2024-03-27T08:13:06.263 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-27T08:13:06.263 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881 --owner scheduled_matan@teuthology --verbose --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --name matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi -- /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/orig.config.yaml 2024-03-27T08:13:06.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T08:13:06.377 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881 branch: wip-matanb-crimson-foreign-ptr description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7625881' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-foreign-ptr ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: eae914f097640d21427554e7f6aa6f1e9c4a7748 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 7338 sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: eae914f097640d21427554e7f6aa6f1e9c4a7748 targets: smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVyO7I+u8QQoEIR1UvSZVM4yfR+p9m5dh/TewU1IX0dTmvsTWenBaVLx8NHfIV/mDD0qBDszdiLSUURv4ZfSfc= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+RDlkkGXbTGmZGsjfG3RBy0+cKR5y1laHPk2EWs5LpMJ+c0Do2UyQm7JiByO/HcdIeY5HRPYduZtEUnASxpn0= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_08:05:55 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-27T08:13:06.377 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa; will attempt to use it 2024-03-27T08:13:06.378 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa/tasks 2024-03-27T08:13:06.378 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-27T08:13:06.380 INFO:teuthology.task.internal:Checking packages... 2024-03-27T08:13:06.404 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '022fc18aa97a30a2f3803bcc4f7d8f3544371a09' 2024-03-27T08:13:06.404 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-27T08:13:06.404 INFO:teuthology.packaging:ref: None 2024-03-27T08:13:06.404 INFO:teuthology.packaging:tag: None 2024-03-27T08:13:06.404 INFO:teuthology.packaging:branch: wip-matanb-crimson-foreign-ptr 2024-03-27T08:13:06.404 INFO:teuthology.packaging:sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:13:06.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-foreign-ptr 2024-03-27T08:13:06.621 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2463.g022fc18a 2024-03-27T08:13:06.623 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-27T08:13:06.630 INFO:teuthology.task.internal:no buildpackages task found 2024-03-27T08:13:06.630 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-27T08:13:06.694 INFO:teuthology.task.internal:Saving configuration 2024-03-27T08:13:06.706 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-27T08:13:06.733 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T08:13:06.758 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:08:02.636494', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVyO7I+u8QQoEIR1UvSZVM4yfR+p9m5dh/TewU1IX0dTmvsTWenBaVLx8NHfIV/mDD0qBDszdiLSUURv4ZfSfc='} 2024-03-27T08:13:06.782 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:08:02.638529', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+RDlkkGXbTGmZGsjfG3RBy0+cKR5y1laHPk2EWs5LpMJ+c0Do2UyQm7JiByO/HcdIeY5HRPYduZtEUnASxpn0='} 2024-03-27T08:13:06.782 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-27T08:13:06.789 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-03-27T08:13:06.789 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-27T08:13:06.789 INFO:teuthology.run_tasks:Running task console_log... 2024-03-27T08:13:06.868 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7bd4bcef70>, signals=[15]) 2024-03-27T08:13:06.869 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-27T08:13:06.877 INFO:teuthology.task.internal:Opening connections... 2024-03-27T08:13:06.877 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:13:06.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:13:06.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:13:06.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:13:07.058 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-27T08:13:07.064 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-03-27T08:13:07.082 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-03-27T08:13:07.082 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-03-27T08:13:07.138 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-03-27T08:13:07.138 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-03-27T08:13:07.138 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-03-27T08:13:07.138 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T08:13:07.139 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T08:13:07.140 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-03-27T08:13:07.163 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-03-27T08:13:07.180 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-03-27T08:13:07.180 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-03-27T08:13:07.240 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-03-27T08:13:07.240 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-03-27T08:13:07.240 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-03-27T08:13:07.240 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T08:13:07.241 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T08:13:07.241 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-03-27T08:13:07.263 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-27T08:13:07.271 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-27T08:13:07.322 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-27T08:13:07.322 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-03-27T08:13:07.325 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-03-27T08:13:07.341 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-27T08:13:07.373 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-27T08:13:07.373 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-03-27T08:13:07.382 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-03-27T08:13:07.498 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-27T08:13:07.555 INFO:teuthology.run_tasks:Running task kernel... 2024-03-27T08:13:07.569 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-27T08:13:07.570 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-27T08:13:07.570 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T08:13:07.571 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T08:13:07.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:13:07.587 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-03-27T08:13:07.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:13:07.588 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-03-27T08:13:07.643 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-361.el9.x86_64 2024-03-27T08:13:07.644 INFO:teuthology.task.kernel:Running kernel on smithi104: 5.14.0-361.el9.x86_64 2024-03-27T08:13:07.644 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum install -y kernel 2024-03-27T08:13:07.649 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-361.el9.x86_64 2024-03-27T08:13:07.649 INFO:teuthology.task.kernel:Running kernel on smithi177: 5.14.0-361.el9.x86_64 2024-03-27T08:13:07.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2024-03-27T08:13:09.658 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-27T08:13:09.773 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-27T08:13:11.274 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-27T08:13:11.448 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-27T08:13:13.549 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-03-27T08:13:16.206 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-27T08:13:16.748 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:05 2024-03-27T08:13:21.681 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.1 MB 00:01 2024-03-27T08:13:22.981 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 16 kB 00:00 2024-03-27T08:13:24.125 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-27T08:13:29.057 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-27T08:13:31.197 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-27T08:13:31.198 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-27T08:13:31.299 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:13:31.315 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:13:31.315 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-03-27T08:13:31.315 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout:Install 4 Packages 2024-03-27T08:13:31.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:13:31.317 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 76 M 2024-03-27T08:13:31.317 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 123 M 2024-03-27T08:13:31.317 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:13:31.533 INFO:teuthology.orchestra.run.smithi104.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 612 kB/s | 66 kB 00:00 2024-03-27T08:13:31.859 INFO:teuthology.orchestra.run.smithi104.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 34 MB/s | 15 MB 00:00 2024-03-27T08:13:32.259 INFO:teuthology.orchestra.run.smithi104.stdout:(3/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 40 MB/s | 33 MB 00:00 2024-03-27T08:13:32.643 INFO:teuthology.orchestra.run.smithi104.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 25 MB/s | 27 MB 00:01 2024-03-27T08:13:32.643 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:13:32.643 INFO:teuthology.orchestra.run.smithi104.stdout:Total 57 MB/s | 76 MB 00:01 2024-03-27T08:13:33.310 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:13:33.668 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:13:33.668 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:13:34.653 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:13:34.653 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:13:37.111 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:13:37.714 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:13:37.808 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:13:40.543 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:13:40.675 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:13:45.390 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:13:45.541 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:50.254 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:14:19.431 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:14:19.431 INFO:teuthology.orchestra.run.smithi104.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T08:14:19.431 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:14:19.475 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:14:19.883 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:14:19.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:14:19.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:14:19.884 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:14:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:14:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:14:20.607 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:14:20.710 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:14:20.711 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:14:20.711 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-27T08:14:20.711 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-27T08:14:20.711 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-27T08:14:20.711 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-27T08:14:20.712 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi104.front.sepia.ceph.com, path=None, version=distro) 2024-03-27T08:14:20.712 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:14:20.791 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:14:20.792 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:14:20.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo rpm -qi grub2-tools 2024-03-27T08:14:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:Name : grub2-tools 2024-03-27T08:14:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:Epoch : 1 2024-03-27T08:14:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:Version : 2.06 2024-03-27T08:14:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:Release : 68.el9 2024-03-27T08:14:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: x86_64 2024-03-27T08:14:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Group : Unspecified 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Size : 8264254 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:License : GPLv3+ 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Packager : builder@centos.org 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Vendor : CentOS 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Summary : Support tools for GRUB. 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:Description : 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:14:20.827 INFO:teuthology.orchestra.run.smithi104.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T08:14:20.828 INFO:teuthology.orchestra.run.smithi104.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T08:14:20.828 INFO:teuthology.orchestra.run.smithi104.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T08:14:20.828 INFO:teuthology.orchestra.run.smithi104.stdout:hardware devices. 2024-03-27T08:14:20.828 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:14:20.828 INFO:teuthology.orchestra.run.smithi104.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T08:14:20.829 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T08:14:20.829 INFO:teuthology.task.kernel:Updating grub on smithi104 to boot 5.14.0-432.el9.x86_64 2024-03-27T08:14:20.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T08:14:21.408 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-03-27T08:14:22.888 INFO:teuthology.orchestra.run.smithi104.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T08:14:22.918 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-03-27T08:14:22.920 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-03-27T08:14:22.935 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.qSfkjhWYzn 2024-03-27T08:14:22.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qSfkjhWYzn 2024-03-27T08:14:23.003 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.qSfkjhWYzn 2024-03-27T08:14:23.143 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.qSfkjhWYzn is 6KB 2024-03-27T08:14:23.192 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.qSfkjhWYzn 2024-03-27T08:14:23.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T08:14:23.271 INFO:teuthology.orchestra.run.smithi104.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T08:14:23.271 INFO:teuthology.orchestra.run.smithi104.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T08:14:23.271 INFO:teuthology.orchestra.run.smithi104.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T08:14:23.271 INFO:teuthology.orchestra.run.smithi104.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-27T08:14:23.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-27T08:14:23.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo shutdown -r now 2024-03-27T08:14:44.027 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 224 kB/s | 19 MB 01:26 2024-03-27T08:14:50.169 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.1 MB 00:02 2024-03-27T08:14:51.770 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-03-27T08:14:53.249 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-27T08:14:53.423 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-27T08:14:53.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-03-27T08:14:53.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:14:58.154 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-03-27T08:15:00.302 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-27T08:15:00.303 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-27T08:15:00.390 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:15:00.402 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:15:00.402 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-03-27T08:15:00.402 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout:Install 4 Packages 2024-03-27T08:15:00.403 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:15:00.404 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 76 M 2024-03-27T08:15:00.404 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 123 M 2024-03-27T08:15:00.404 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:15:00.880 INFO:teuthology.orchestra.run.smithi177.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 184 kB/s | 66 kB 00:00 2024-03-27T08:15:03.272 INFO:teuthology.orchestra.run.smithi177.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 5.4 MB/s | 15 MB 00:02 2024-03-27T08:15:11.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-03-27T08:15:20.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-03-27T08:15:20.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:15:20.945 INFO:teuthology.orchestra.run.smithi177.stdout:(3/4): kernel-modules-core-5.14.0-432.el9.x86_6 1.4 MB/s | 27 MB 00:20 2024-03-27T08:15:23.445 INFO:teuthology.orchestra.run.smithi177.stdout:(4/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 1.5 MB/s | 33 MB 00:22 2024-03-27T08:15:23.445 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:15:23.445 INFO:teuthology.orchestra.run.smithi177.stdout:Total 3.3 MB/s | 76 MB 00:23 2024-03-27T08:15:23.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-03-27T08:15:24.103 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:15:24.481 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:15:24.482 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:15:25.488 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:15:25.489 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:15:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:15:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:15:28.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:15:31.764 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:15:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:15:35.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-03-27T08:15:35.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:15:36.605 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:15:36.639 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:15:41.363 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:16:07.804 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-03-27T08:16:08.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-03-27T08:16:08.201 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-432.el9.x86_64"... 2024-03-27T08:16:08.201 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-03-27T08:16:08.214 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-432.el9.x86_64 2024-03-27T08:16:08.215 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-27T08:16:08.215 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T08:16:08.215 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T08:16:09.216 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-27T08:16:09.216 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T08:16:09.314 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2024-03-27T08:16:09.356 DEBUG:teuthology.parallel:result is None 2024-03-27T08:16:10.482 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:16:10.482 INFO:teuthology.orchestra.run.smithi177.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T08:16:10.482 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:16:10.526 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:16:10.924 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:16:10.924 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:16:10.924 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:16:10.924 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:16:11.336 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:16:11.656 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:16:11.773 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:16:11.773 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:16:11.773 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-27T08:16:11.773 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-27T08:16:11.773 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-27T08:16:11.773 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-03-27T08:16:11.773 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2024-03-27T08:16:11.774 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:16:11.853 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:16:11.854 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:16:11.854 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.06 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 68.el9 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Group : Unspecified 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 8264254 2024-03-27T08:16:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : builder@centos.org 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2024-03-27T08:16:11.898 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:16:11.899 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T08:16:11.899 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T08:16:11.899 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 5.14.0-432.el9.x86_64 2024-03-27T08:16:11.900 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T08:16:12.530 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-03-27T08:16:14.001 INFO:teuthology.orchestra.run.smithi177.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T08:16:14.027 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-03-27T08:16:14.035 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-03-27T08:16:14.067 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.AlJdplxtIC 2024-03-27T08:16:14.067 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AlJdplxtIC 2024-03-27T08:16:14.143 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.AlJdplxtIC 2024-03-27T08:16:14.302 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.AlJdplxtIC is 6KB 2024-03-27T08:16:14.336 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.AlJdplxtIC 2024-03-27T08:16:14.354 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T08:16:14.423 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T08:16:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T08:16:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T08:16:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-27T08:16:14.425 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-27T08:16:14.599 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-03-27T08:16:44.603 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-03-27T08:16:44.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:16:44.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:17:03.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-27T08:17:12.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:17:12.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:17:15.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-27T08:17:27.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:17:27.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:17:45.503 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-27T08:18:00.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:18:00.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:18:00.931 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-03-27T08:18:01.361 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:18:01.361 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-432.el9.x86_64"... 2024-03-27T08:18:01.361 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-03-27T08:18:01.375 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-432.el9.x86_64 2024-03-27T08:18:01.375 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-27T08:18:01.376 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T08:18:01.376 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T08:18:02.376 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-03-27T08:18:02.377 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T08:18:02.438 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2024-03-27T08:18:02.472 DEBUG:teuthology.parallel:result is None 2024-03-27T08:18:02.472 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-27T08:18:02.481 INFO:teuthology.task.internal:Creating test directory... 2024-03-27T08:18:02.481 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T08:18:02.485 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T08:18:02.500 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-27T08:18:02.523 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-27T08:18:02.570 INFO:teuthology.task.internal:Creating archive directory... 2024-03-27T08:18:02.570 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T08:18:02.572 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T08:18:02.599 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-27T08:18:02.610 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-27T08:18:02.610 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T08:18:02.639 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T08:18:02.681 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:18:02.688 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:18:02.695 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:18:02.701 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:18:02.702 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-27T08:18:02.709 INFO:teuthology.task.internal:Configuring sudo... 2024-03-27T08:18:02.709 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T08:18:02.739 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T08:18:02.766 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-27T08:18:02.775 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-27T08:18:02.776 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T08:18:02.809 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T08:18:02.824 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:18:02.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:18:02.962 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:18:03.046 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:18:03.090 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:18:03.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T08:18:03.160 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:18:03.202 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:18:03.253 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:18:03.332 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:18:03.371 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:18:03.371 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T08:18:03.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-03-27T08:18:03.438 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-03-27T08:18:03.483 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:18:03.529 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:18:03.965 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-27T08:18:03.974 INFO:teuthology.task.internal:Starting timer... 2024-03-27T08:18:03.974 INFO:teuthology.run_tasks:Running task pcp... 2024-03-27T08:18:04.021 INFO:teuthology.run_tasks:Running task selinux... 2024-03-27T08:18:04.078 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-27T08:18:04.079 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-03-27T08:18:04.135 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-03-27T08:18:04.137 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-03-27T08:18:04.252 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-03-27T08:18:04.254 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-27T08:18:04.254 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-03-27T08:18:04.283 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-03-27T08:18:04.283 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-03-27T08:18:04.306 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-03-27T08:18:04.306 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2024-03-27T08:18:04.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:18:04.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:18:04.353 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:18:04.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:18:04.401 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-27T08:18:04.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-03-27T08:18:04.432 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-03-27T08:18:04.684 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-27T08:18:04.693 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-27T08:18:04.709 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-27T08:18:04.710 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi104.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-27T08:22:30.459 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-03-27T08:22:30.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-03-27T08:22:30.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:22:30.553 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-03-27T08:22:30.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-03-27T08:22:30.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:22:30.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:22:30.722 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-03-27T08:22:30.805 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T08:22:30.805 INFO:teuthology.run_tasks:Running task clock... 2024-03-27T08:22:30.816 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-27T08:22:30.816 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T08:22:30.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:22:30.819 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T08:22:30.819 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:22:30.849 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T08:22:30.866 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T08:22:30.876 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T08:22:30.893 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T08:22:30.896 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-03-27T08:22:30.910 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-03-27T08:22:30.923 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-03-27T08:22:30.926 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T08:22:30.936 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-03-27T08:22:30.942 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T08:22:30.953 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T08:22:30.969 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T08:22:30.999 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:22:31.002 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:22:31.003 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-03-27T08:22:31.003 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.003 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.003 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.003 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.027 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:22:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:22:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-03-27T08:22:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:22:31.032 INFO:teuthology.run_tasks:Running task install... 2024-03-27T08:22:31.041 DEBUG:teuthology.task.install:project ceph 2024-03-27T08:22:31.041 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'}} 2024-03-27T08:22:31.042 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'} 2024-03-27T08:22:31.042 INFO:teuthology.task.install:Using flavor: crimson 2024-03-27T08:22:31.048 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-27T08:22:31.048 INFO:teuthology.task.install:extra packages: [] 2024-03-27T08:22:31.048 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'tag': None, 'wait_for_package': False} 2024-03-27T08:22:31.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:22:31.051 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'tag': None, 'wait_for_package': False} 2024-03-27T08:22:31.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:22:31.195 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/ 2024-03-27T08:22:31.195 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2463.g022fc18a 2024-03-27T08:22:31.199 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/ 2024-03-27T08:22:31.199 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2463.g022fc18a 2024-03-27T08:22:31.410 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T08:22:31.410 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:22:31.410 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T08:22:31.438 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-27T08:22:31.438 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T08:22:31.479 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T08:22:31.479 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:22:31.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T08:22:31.506 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T08:22:31.508 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-27T08:22:31.508 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T08:22:31.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T08:22:31.586 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T08:22:31.652 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-03-27T08:22:31.653 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-03-27T08:22:31.659 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T08:22:31.726 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-03-27T08:22:31.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-03-27T08:22:31.904 INFO:teuthology.orchestra.run.smithi177.stdout:74 files removed 2024-03-27T08:22:31.932 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw 2024-03-27T08:22:31.971 INFO:teuthology.orchestra.run.smithi104.stdout:74 files removed 2024-03-27T08:22:31.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw 2024-03-27T08:22:32.807 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 151 kB/s | 87 kB 00:00 2024-03-27T08:22:33.037 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 117 kB/s | 87 kB 00:00 2024-03-27T08:22:33.466 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-03-27T08:22:33.696 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-03-27T08:22:34.042 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 3.8 kB/s | 2.0 kB 00:00 2024-03-27T08:22:34.288 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 3.7 kB/s | 2.0 kB 00:00 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-27T08:22:34.857 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-27T08:22:35.969 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-27T08:22:36.365 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-27T08:22:37.622 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-03-27T08:22:39.811 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-27T08:22:43.701 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-27T08:22:45.085 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-03-27T08:22:45.142 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 933 kB/s | 8.0 MB 00:08 2024-03-27T08:22:47.553 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-03-27T08:22:52.445 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-03-27T08:22:52.567 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 19 MB 00:06 2024-03-27T08:22:54.884 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:22:54.887 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 9.6 M 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.7 M 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T08:22:54.888 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.9 M 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 19 M 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 25 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 796 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 548 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:22:54.889 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 45 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 134 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 159 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 315 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 293 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 99 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T08:22:54.890 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout:Install 30 Packages 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 2 Packages 2024-03-27T08:22:54.891 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:22:54.892 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 58 M 2024-03-27T08:22:54.892 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:22:56.504 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-27T08:22:56.703 INFO:teuthology.orchestra.run.smithi104.stdout:(1/32): ceph-base-19.0.0-2463.g022fc18a.el9.x86 3.6 MB/s | 5.9 MB 00:01 2024-03-27T08:22:56.745 INFO:teuthology.orchestra.run.smithi104.stdout:(2/32): ceph-selinux-19.0.0-2463.g022fc18a.el9. 604 kB/s | 25 kB 00:00 2024-03-27T08:22:56.945 INFO:teuthology.orchestra.run.smithi104.stdout:(3/32): libcephfs2-19.0.0-2463.g022fc18a.el9.x8 3.9 MB/s | 796 kB 00:00 2024-03-27T08:22:57.204 INFO:teuthology.orchestra.run.smithi104.stdout:(4/32): ceph-radosgw-19.0.0-2463.g022fc18a.el9. 4.5 MB/s | 9.6 MB 00:02 2024-03-27T08:22:57.229 INFO:teuthology.orchestra.run.smithi104.stdout:(5/32): libradosstriper1-19.0.0-2463.g022fc18a. 1.9 MB/s | 548 kB 00:00 2024-03-27T08:22:57.330 INFO:teuthology.orchestra.run.smithi104.stdout:(6/32): python3-ceph-argparse-19.0.0-2463.g022f 450 kB/s | 45 kB 00:00 2024-03-27T08:22:57.430 INFO:teuthology.orchestra.run.smithi104.stdout:(7/32): python3-ceph-common-19.0.0-2463.g022fc1 1.3 MB/s | 134 kB 00:00 2024-03-27T08:22:57.530 INFO:teuthology.orchestra.run.smithi104.stdout:(8/32): python3-cephfs-19.0.0-2463.g022fc18a.el 1.6 MB/s | 159 kB 00:00 2024-03-27T08:22:57.655 INFO:teuthology.orchestra.run.smithi104.stdout:(9/32): python3-rados-19.0.0-2463.g022fc18a.el9 2.5 MB/s | 315 kB 00:00 2024-03-27T08:22:57.764 INFO:teuthology.orchestra.run.smithi104.stdout:(10/32): librgw2-19.0.0-2463.g022fc18a.el9.x86_ 9.0 MB/s | 5.0 MB 00:00 2024-03-27T08:22:57.789 INFO:teuthology.orchestra.run.smithi104.stdout:(11/32): python3-rbd-19.0.0-2463.g022fc18a.el9. 2.1 MB/s | 293 kB 00:00 2024-03-27T08:22:57.840 INFO:teuthology.orchestra.run.smithi104.stdout:(12/32): python3-rgw-19.0.0-2463.g022fc18a.el9. 1.3 MB/s | 99 kB 00:00 2024-03-27T08:22:57.907 INFO:teuthology.orchestra.run.smithi104.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 906 kB/s | 106 kB 00:00 2024-03-27T08:22:57.932 INFO:teuthology.orchestra.run.smithi104.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-03-27T08:22:57.957 INFO:teuthology.orchestra.run.smithi104.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 889 kB/s | 44 kB 00:00 2024-03-27T08:22:58.016 INFO:teuthology.orchestra.run.smithi104.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-27T08:22:58.041 INFO:teuthology.orchestra.run.smithi104.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-27T08:22:58.066 INFO:teuthology.orchestra.run.smithi104.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 828 kB/s | 41 kB 00:00 2024-03-27T08:22:58.158 INFO:teuthology.orchestra.run.smithi104.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 391 kB/s | 36 kB 00:00 2024-03-27T08:22:58.225 INFO:teuthology.orchestra.run.smithi104.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-03-27T08:22:58.250 INFO:teuthology.orchestra.run.smithi104.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-03-27T08:22:58.296 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-27T08:22:58.951 INFO:teuthology.orchestra.run.smithi104.stdout:(22/32): ceph-common-19.0.0-2463.g022fc18a.el9. 4.9 MB/s | 19 MB 00:03 2024-03-27T08:22:58.984 INFO:teuthology.orchestra.run.smithi104.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 407 kB/s | 309 kB 00:00 2024-03-27T08:22:59.109 INFO:teuthology.orchestra.run.smithi104.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi104.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-03-27T08:22:59.368 INFO:teuthology.orchestra.run.smithi104.stdout:(26/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 62 kB/s | 26 kB 00:00 2024-03-27T08:22:59.535 INFO:teuthology.orchestra.run.smithi104.stdout:(27/32): libarrow-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-03-27T08:22:59.585 INFO:teuthology.orchestra.run.smithi104.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-03-27T08:22:59.711 INFO:teuthology.orchestra.run.smithi104.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-03-27T08:22:59.795 INFO:teuthology.orchestra.run.smithi104.stdout:(30/32): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-03-27T08:23:00.006 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-27T08:23:00.078 INFO:teuthology.orchestra.run.smithi104.stdout:(31/32): librados2-19.0.0-2463.g022fc18a.el9.x8 6.9 MB/s | 3.4 MB 00:00 2024-03-27T08:23:00.253 INFO:teuthology.orchestra.run.smithi104.stdout:(32/32): librbd1-19.0.0-2463.g022fc18a.el9.x86_ 6.8 MB/s | 3.7 MB 00:00 2024-03-27T08:23:00.257 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:23:00.257 INFO:teuthology.orchestra.run.smithi104.stdout:Total 11 MB/s | 58 MB 00:05 2024-03-27T08:23:00.289 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:23:00.324 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:23:00.324 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:23:00.796 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:23:00.796 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:23:02.256 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:23:02.726 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-27T08:23:02.763 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:23:02.838 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:23:02.933 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:23:02.957 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:23:03.019 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:23:03.056 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-03-27T08:23:03.085 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-03-27T08:23:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-03-27T08:23:03.159 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:23:03.175 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:23:03.375 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:23:03.408 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:23:03.758 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-03-27T08:23:03.788 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:23:03.838 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:23:03.880 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 12/34 2024-03-27T08:23:03.934 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:23:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 14/34 2024-03-27T08:23:04.038 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-03-27T08:23:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:23:04.154 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:23:04.189 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-03-27T08:23:04.295 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-03-27T08:23:04.332 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-03-27T08:23:04.387 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-03-27T08:23:04.422 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-03-27T08:23:04.450 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-03-27T08:23:04.486 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-03-27T08:23:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-27T08:23:04.988 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-27T08:23:05.074 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-27T08:23:05.096 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:23:05.173 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:23:05.436 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 27/34 2024-03-27T08:23:05.466 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 28/34 2024-03-27T08:23:07.421 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:23:07.432 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:23:07.432 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:23:07.432 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:23:07.432 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 9.6 M 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.7 M 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.9 M 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 19 M 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 25 k 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T08:23:07.433 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 796 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 548 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 45 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 134 k 2024-03-27T08:23:07.434 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 159 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 315 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 293 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 99 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:23:07.435 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:23:07.436 INFO:teuthology.orchestra.run.smithi177.stdout:Install 30 Packages 2024-03-27T08:23:07.436 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 2 Packages 2024-03-27T08:23:07.436 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:07.436 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 58 M 2024-03-27T08:23:07.436 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:23:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:23:07.879 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:23:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:23:08.615 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:23:08.709 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:23:08.709 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T08:23:08.709 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T08:23:08.710 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:08.749 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:23:10.364 INFO:teuthology.orchestra.run.smithi177.stdout:(1/32): ceph-radosgw-19.0.0-2463.g022fc18a.el9. 3.8 MB/s | 9.6 MB 00:02 2024-03-27T08:23:10.456 INFO:teuthology.orchestra.run.smithi177.stdout:(2/32): ceph-selinux-19.0.0-2463.g022fc18a.el9. 275 kB/s | 25 kB 00:00 2024-03-27T08:23:10.615 INFO:teuthology.orchestra.run.smithi177.stdout:(3/32): libcephfs2-19.0.0-2463.g022fc18a.el9.x8 4.9 MB/s | 796 kB 00:00 2024-03-27T08:23:10.723 INFO:teuthology.orchestra.run.smithi177.stdout:(4/32): libradosstriper1-19.0.0-2463.g022fc18a. 5.0 MB/s | 548 kB 00:00 2024-03-27T08:23:10.849 INFO:teuthology.orchestra.run.smithi177.stdout:(5/32): ceph-base-19.0.0-2463.g022fc18a.el9.x86 2.0 MB/s | 5.9 MB 00:03 2024-03-27T08:23:10.958 INFO:teuthology.orchestra.run.smithi177.stdout:(6/32): python3-ceph-argparse-19.0.0-2463.g022f 417 kB/s | 45 kB 00:00 2024-03-27T08:23:11.083 INFO:teuthology.orchestra.run.smithi177.stdout:(7/32): python3-ceph-common-19.0.0-2463.g022fc1 1.0 MB/s | 134 kB 00:00 2024-03-27T08:23:11.191 INFO:teuthology.orchestra.run.smithi177.stdout:(8/32): python3-cephfs-19.0.0-2463.g022fc18a.el 1.4 MB/s | 159 kB 00:00 2024-03-27T08:23:11.325 INFO:teuthology.orchestra.run.smithi177.stdout:(9/32): librgw2-19.0.0-2463.g022fc18a.el9.x86_6 8.4 MB/s | 5.0 MB 00:00 2024-03-27T08:23:11.359 INFO:teuthology.orchestra.run.smithi177.stdout:(10/32): python3-rados-19.0.0-2463.g022fc18a.el 1.8 MB/s | 315 kB 00:00 2024-03-27T08:23:11.576 INFO:teuthology.orchestra.run.smithi177.stdout:(11/32): ceph-common-19.0.0-2463.g022fc18a.el9. 5.1 MB/s | 19 MB 00:03 2024-03-27T08:23:11.601 INFO:teuthology.orchestra.run.smithi177.stdout:(12/32): python3-rgw-19.0.0-2463.g022fc18a.el9. 410 kB/s | 99 kB 00:00 2024-03-27T08:23:11.651 INFO:teuthology.orchestra.run.smithi177.stdout:(13/32): python3-rbd-19.0.0-2463.g022fc18a.el9. 899 kB/s | 293 kB 00:00 2024-03-27T08:23:11.718 INFO:teuthology.orchestra.run.smithi177.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi177.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 44 kB 00:00 2024-03-27T08:23:11.802 INFO:teuthology.orchestra.run.smithi177.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 909 kB/s | 159 kB 00:00 2024-03-27T08:23:11.852 INFO:teuthology.orchestra.run.smithi177.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 661 kB 00:00 2024-03-27T08:23:11.886 INFO:teuthology.orchestra.run.smithi177.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 282 kB 00:00 2024-03-27T08:23:11.911 INFO:teuthology.orchestra.run.smithi177.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 380 kB/s | 41 kB 00:00 2024-03-27T08:23:12.003 INFO:teuthology.orchestra.run.smithi177.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 392 kB/s | 36 kB 00:00 2024-03-27T08:23:12.028 INFO:teuthology.orchestra.run.smithi177.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-03-27T08:23:12.062 INFO:teuthology.orchestra.run.smithi177.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-03-27T08:23:12.487 INFO:teuthology.orchestra.run.smithi177.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 61 kB/s | 26 kB 00:00 2024-03-27T08:23:12.687 INFO:teuthology.orchestra.run.smithi177.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 243 kB/s | 49 kB 00:00 2024-03-27T08:23:12.721 INFO:teuthology.orchestra.run.smithi177.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 430 kB/s | 309 kB 00:00 2024-03-27T08:23:12.846 INFO:teuthology.orchestra.run.smithi177.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-03-27T08:23:13.030 INFO:teuthology.orchestra.run.smithi177.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-03-27T08:23:13.105 INFO:teuthology.orchestra.run.smithi177.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 741 kB/s | 192 kB 00:00 2024-03-27T08:23:13.172 INFO:teuthology.orchestra.run.smithi177.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-03-27T08:23:13.280 INFO:teuthology.orchestra.run.smithi177.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-27T08:23:13.589 INFO:teuthology.orchestra.run.smithi177.stdout:(31/32): librbd1-19.0.0-2463.g022fc18a.el9.x86_ 8.8 MB/s | 3.7 MB 00:00 2024-03-27T08:23:13.881 INFO:teuthology.orchestra.run.smithi177.stdout:(32/32): librados2-19.0.0-2463.g022fc18a.el9.x8 4.4 MB/s | 3.4 MB 00:00 2024-03-27T08:23:13.887 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:23:13.887 INFO:teuthology.orchestra.run.smithi177.stdout:Total 9.0 MB/s | 58 MB 00:06 2024-03-27T08:23:13.921 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:23:13.960 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:23:13.961 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:23:14.423 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:23:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:23:15.899 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:23:16.387 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-27T08:23:16.425 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:23:16.499 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:23:16.592 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:23:16.618 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:23:16.672 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:23:16.717 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-03-27T08:23:16.747 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-03-27T08:23:16.772 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-03-27T08:23:16.795 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:23:16.810 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:23:16.995 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:23:17.029 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:23:17.386 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-03-27T08:23:17.424 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:23:17.474 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:23:17.525 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 12/34 2024-03-27T08:23:17.598 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:23:17.630 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 14/34 2024-03-27T08:23:17.718 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-03-27T08:23:17.752 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:23:17.824 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:23:17.859 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-03-27T08:23:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-03-27T08:23:18.002 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-03-27T08:23:18.042 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-03-27T08:23:18.074 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-03-27T08:23:18.103 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-03-27T08:23:18.135 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-03-27T08:23:18.195 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-27T08:23:18.533 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:23:18.533 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-03-27T08:23:18.533 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-03-27T08:23:18.533 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-03-27T08:23:18.533 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-03-27T08:23:18.533 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-03-27T08:23:18.534 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-03-27T08:23:18.534 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-03-27T08:23:18.534 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-27T08:23:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:23:18.855 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:23:18.932 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 27/34 2024-03-27T08:23:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 28/34 2024-03-27T08:23:19.552 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:23:19.576 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:23:19.576 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:23:19.576 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T08:23:19.577 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:23:19.577 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:23:19.577 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:19.578 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:23:19.655 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:23:19.655 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 1/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 5/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 6/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 7/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 8/34 2024-03-27T08:23:20.872 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 9/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 10/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 12/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-27T08:23:20.873 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-27T08:23:20.874 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-27T08:23:20.874 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-27T08:23:20.874 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-27T08:23:20.874 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-27T08:23:20.874 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-27T08:23:20.875 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-27T08:23:20.875 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:23:20.875 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-27T08:23:20.875 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 33/34 2024-03-27T08:23:21.579 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:23:21.615 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:23:22.132 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.133 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T08:23:22.134 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:22.135 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:23:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:23:22.318 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:23:22.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-test 2024-03-27T08:23:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:23:22.423 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T08:23:22.423 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T08:23:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:22.461 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:23:22.957 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:30 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:23:23.108 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:23:23.109 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:23:23.109 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:23:23.109 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:23:23.109 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 87 M 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 174 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-03-27T08:23:23.110 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:Install 11 Packages 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 90 M 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 819 M 2024-03-27T08:23:23.111 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:23:23.950 INFO:teuthology.orchestra.run.smithi104.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-27T08:23:24.025 INFO:teuthology.orchestra.run.smithi104.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-03-27T08:23:24.067 INFO:teuthology.orchestra.run.smithi104.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-27T08:23:24.100 INFO:teuthology.orchestra.run.smithi104.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-27T08:23:24.134 INFO:teuthology.orchestra.run.smithi104.stdout:(5/11): libcephsqlite-19.0.0-2463.g022fc18a.el9 488 kB/s | 174 kB 00:00 2024-03-27T08:23:24.159 INFO:teuthology.orchestra.run.smithi104.stdout:(6/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.5 MB/s | 90 kB 00:00 2024-03-27T08:23:24.301 INFO:teuthology.orchestra.run.smithi104.stdout:(7/11): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-27T08:23:25.160 INFO:teuthology.orchestra.run.smithi104.stdout:(8/11): libubsan-11.4.1-3.el9.x86_64.rpm 218 kB/s | 187 kB 00:00 2024-03-27T08:23:25.494 INFO:teuthology.orchestra.run.smithi104.stdout:(9/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 351 kB/s | 469 kB 00:01 2024-03-27T08:23:25.685 INFO:teuthology.orchestra.run.smithi104.stdout:(10/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 232 kB/s | 122 kB 00:00 2024-03-27T08:23:27.977 INFO:teuthology.orchestra.run.smithi104.stdout:(11/11): ceph-test-19.0.0-2463.g022fc18a.el9.x8 21 MB/s | 87 MB 00:04 2024-03-27T08:23:27.980 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:23:27.980 INFO:teuthology.orchestra.run.smithi104.stdout:Total 18 MB/s | 90 MB 00:04 2024-03-27T08:23:28.022 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:23:28.046 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:23:28.081 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:23:28.687 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:23:28.687 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:23:29.730 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:23:29.796 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-03-27T08:23:29.828 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-03-27T08:23:29.854 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:23:29.925 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:23:29.953 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-03-27T08:23:30.008 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-03-27T08:23:30.050 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-03-27T08:23:30.082 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-03-27T08:23:30.121 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-03-27T08:23:30.166 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-03-27T08:23:30.192 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:23:32.302 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:23:32.302 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-03-27T08:23:32.303 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:33.328 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:23:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:23:33.353 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:23:33.353 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T08:23:33.353 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:23:33.353 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:23:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:23:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:23:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 1/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 5/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 6/34 2024-03-27T08:23:34.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 7/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 8/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 9/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 10/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 12/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-03-27T08:23:34.682 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-27T08:23:34.683 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-27T08:23:34.684 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:23:34.684 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-27T08:23:34.684 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 33/34 2024-03-27T08:23:35.703 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:23:35.703 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:35.703 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-03-27T08:23:35.703 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.704 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.705 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:35.706 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:23:36.042 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-test 2024-03-27T08:23:36.618 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:32 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:23:36.779 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 87 M 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 174 k 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-03-27T08:23:36.780 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout:Install 11 Packages 2024-03-27T08:23:36.781 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:23:36.782 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 90 M 2024-03-27T08:23:36.782 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 819 M 2024-03-27T08:23:36.782 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:23:37.667 INFO:teuthology.orchestra.run.smithi177.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 644 kB/s | 217 kB 00:00 2024-03-27T08:23:37.767 INFO:teuthology.orchestra.run.smithi177.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-03-27T08:23:37.809 INFO:teuthology.orchestra.run.smithi177.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-27T08:23:37.842 INFO:teuthology.orchestra.run.smithi177.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-27T08:23:37.876 INFO:teuthology.orchestra.run.smithi177.stdout:(5/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.7 MB/s | 90 kB 00:00 2024-03-27T08:23:38.018 INFO:teuthology.orchestra.run.smithi177.stdout:(6/11): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-27T08:23:38.102 INFO:teuthology.orchestra.run.smithi177.stdout:(7/11): libcephsqlite-19.0.0-2463.g022fc18a.el9 225 kB/s | 174 kB 00:00 2024-03-27T08:23:38.202 INFO:teuthology.orchestra.run.smithi177.stdout:(8/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 2.5 MB/s | 469 kB 00:00 2024-03-27T08:23:38.236 INFO:teuthology.orchestra.run.smithi177.stdout:(9/11): libubsan-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-03-27T08:23:38.386 INFO:teuthology.orchestra.run.smithi177.stdout:(10/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 664 kB/s | 122 kB 00:00 2024-03-27T08:23:41.858 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:23:41.932 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:23:42.844 INFO:teuthology.orchestra.run.smithi177.stdout:(11/11): ceph-test-19.0.0-2463.g022fc18a.el9.x8 16 MB/s | 87 MB 00:05 2024-03-27T08:23:42.848 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:23:42.848 INFO:teuthology.orchestra.run.smithi177.stdout:Total 15 MB/s | 90 MB 00:06 2024-03-27T08:23:42.893 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:23:42.917 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:23:42.917 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:23:43.555 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:23:43.555 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:23:44.566 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:23:44.648 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-03-27T08:23:44.704 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-03-27T08:23:44.729 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:23:44.798 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:23:44.829 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-03-27T08:23:44.886 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-03-27T08:23:44.934 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-03-27T08:23:44.965 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-03-27T08:23:45.009 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-03-27T08:23:45.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-03-27T08:23:45.078 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:23:48.055 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:23:48.055 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 1/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-03-27T08:23:48.056 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: jq-1.6-16.el9.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: libubsan-11.4.1-3.el9.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T08:23:48.480 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T08:23:48.481 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T08:23:48.481 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T08:23:48.481 INFO:teuthology.orchestra.run.smithi104.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-03-27T08:23:48.481 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:48.481 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:23:48.767 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph 2024-03-27T08:23:49.327 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:57 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:23:49.478 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:23:49.480 INFO:teuthology.orchestra.run.smithi104.stdout:========================================================================================= 2024-03-27T08:23:49.480 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:23:49.480 INFO:teuthology.orchestra.run.smithi104.stdout:========================================================================================= 2024-03-27T08:23:49.480 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:23:49.480 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 6.5 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 2.2 M 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 1.6 M 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 249 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 71 M 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T08:23:49.481 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T08:23:49.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:23:49.483 INFO:teuthology.orchestra.run.smithi104.stdout:========================================================================================= 2024-03-27T08:23:49.484 INFO:teuthology.orchestra.run.smithi104.stdout:Install 36 Packages 2024-03-27T08:23:49.484 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:23:49.484 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 84 M 2024-03-27T08:23:49.484 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 921 M 2024-03-27T08:23:49.484 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:23:50.148 INFO:teuthology.orchestra.run.smithi104.stdout:(1/36): ceph-19.0.0-2463.g022fc18a.el9.x86_64.r 16 kB/s | 6.5 kB 00:00 2024-03-27T08:23:50.540 INFO:teuthology.orchestra.run.smithi104.stdout:(2/36): ceph-mgr-19.0.0-2463.g022fc18a.el9.x86_ 1.9 MB/s | 1.6 MB 00:00 2024-03-27T08:23:51.157 INFO:teuthology.orchestra.run.smithi104.stdout:(3/36): ceph-mds-19.0.0-2463.g022fc18a.el9.x86_ 1.5 MB/s | 2.2 MB 00:01 2024-03-27T08:23:51.282 INFO:teuthology.orchestra.run.smithi104.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2463.g022f 1.9 MB/s | 249 kB 00:00 2024-03-27T08:23:51.408 INFO:teuthology.orchestra.run.smithi104.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-27T08:23:51.449 INFO:teuthology.orchestra.run.smithi104.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-27T08:23:51.483 INFO:teuthology.orchestra.run.smithi104.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 947 kB/s | 32 kB 00:00 2024-03-27T08:23:51.516 INFO:teuthology.orchestra.run.smithi104.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-27T08:23:51.550 INFO:teuthology.orchestra.run.smithi104.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-27T08:23:51.600 INFO:teuthology.orchestra.run.smithi104.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-27T08:23:51.678 INFO:teuthology.orchestra.run.smithi104.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-27T08:23:51.712 INFO:teuthology.orchestra.run.smithi104.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-27T08:23:51.737 INFO:teuthology.orchestra.run.smithi104.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 4.9 MB/s | 124 kB 00:00 2024-03-27T08:23:51.770 INFO:teuthology.orchestra.run.smithi104.stdout:(14/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-03-27T08:23:51.913 INFO:teuthology.orchestra.run.smithi104.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 882 kB/s | 125 kB 00:00 2024-03-27T08:23:51.954 INFO:teuthology.orchestra.run.smithi104.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 537 kB/s | 22 kB 00:00 2024-03-27T08:23:52.379 INFO:teuthology.orchestra.run.smithi104.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 356 kB/s | 151 kB 00:00 2024-03-27T08:23:52.505 INFO:teuthology.orchestra.run.smithi104.stdout:(18/36): ceph-mon-19.0.0-2463.g022fc18a.el9.x86 2.1 MB/s | 5.0 MB 00:02 2024-03-27T08:23:52.613 INFO:teuthology.orchestra.run.smithi104.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 186 kB/s | 43 kB 00:00 2024-03-27T08:23:52.822 INFO:teuthology.orchestra.run.smithi104.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 542 kB/s | 172 kB 00:00 2024-03-27T08:23:52.955 INFO:teuthology.orchestra.run.smithi104.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-03-27T08:23:52.972 INFO:teuthology.orchestra.run.smithi104.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 71 kB/s | 11 kB 00:00 2024-03-27T08:23:53.148 INFO:teuthology.orchestra.run.smithi104.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 92 kB/s | 18 kB 00:00 2024-03-27T08:23:53.181 INFO:teuthology.orchestra.run.smithi104.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 111 kB/s | 23 kB 00:00 2024-03-27T08:23:53.348 INFO:teuthology.orchestra.run.smithi104.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 97 kB/s | 19 kB 00:00 2024-03-27T08:23:53.374 INFO:teuthology.orchestra.run.smithi104.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 102 kB/s | 20 kB 00:00 2024-03-27T08:23:53.582 INFO:teuthology.orchestra.run.smithi104.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 198 kB/s | 46 kB 00:00 2024-03-27T08:23:53.649 INFO:teuthology.orchestra.run.smithi104.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 286 kB/s | 79 kB 00:00 2024-03-27T08:23:53.833 INFO:teuthology.orchestra.run.smithi104.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 90 kB/s | 16 kB 00:00 2024-03-27T08:23:53.899 INFO:teuthology.orchestra.run.smithi104.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 859 kB/s | 272 kB 00:00 2024-03-27T08:23:54.583 INFO:teuthology.orchestra.run.smithi104.stdout:(31/36): ceph-osd-19.0.0-2463.g022fc18a.el9.x86 18 MB/s | 71 MB 00:04 2024-03-27T08:23:54.608 INFO:teuthology.orchestra.run.smithi104.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 116 kB/s | 90 kB 00:00 2024-03-27T08:23:54.633 INFO:teuthology.orchestra.run.smithi104.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 49 kB/s | 36 kB 00:00 2024-03-27T08:23:54.842 INFO:teuthology.orchestra.run.smithi104.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 96 kB/s | 20 kB 00:00 2024-03-27T08:23:54.892 INFO:teuthology.orchestra.run.smithi104.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 745 kB/s | 230 kB 00:00 2024-03-27T08:23:55.018 INFO:teuthology.orchestra.run.smithi104.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.1 MB/s | 427 kB 00:00 2024-03-27T08:23:55.022 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:23:55.022 INFO:teuthology.orchestra.run.smithi104.stdout:Total 15 MB/s | 84 MB 00:05 2024-03-27T08:23:55.047 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:23:55.091 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:23:55.091 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:23:55.810 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:23:55.810 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:23:56.551 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:23:56.624 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:23:57.396 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:23:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-27T08:23:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-27T08:23:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-27T08:23:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-27T08:23:57.629 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-27T08:23:57.662 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-27T08:23:57.753 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-27T08:23:57.823 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-27T08:23:57.864 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-27T08:23:57.935 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-27T08:23:57.989 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-27T08:23:58.028 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-27T08:23:58.077 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-27T08:23:58.132 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-27T08:23:58.317 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-27T08:23:58.366 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-27T08:23:58.440 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-27T08:23:58.472 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-27T08:23:58.502 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-27T08:23:58.578 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-27T08:23:58.612 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:23:58.645 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-27T08:23:58.694 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-27T08:23:58.729 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-27T08:23:58.772 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-27T08:23:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-27T08:23:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-27T08:23:58.983 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-27T08:23:59.119 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-27T08:23:59.237 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:23:59.423 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 31/36 2024-03-27T08:23:59.449 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:23:59.474 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:23:59.474 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:23:59.474 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T08:23:59.474 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:23:59.474 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:23:59.474 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 1/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-03-27T08:24:02.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-03-27T08:24:02.721 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-03-27T08:24:02.721 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-03-27T08:24:02.721 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-03-27T08:24:03.137 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-03-27T08:24:03.137 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:03.137 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:24:03.137 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:03.137 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-16.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: libubsan-11.4.1-3.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:03.138 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:03.437 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph 2024-03-27T08:24:04.001 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:00 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:04.153 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 6.5 k 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 2.2 M 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 1.6 M 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 249 k 2024-03-27T08:24:04.156 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 71 M 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T08:24:04.157 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T08:24:04.158 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T08:24:04.159 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T08:24:04.160 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:04.160 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:24:04.160 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-03-27T08:24:04.160 INFO:teuthology.orchestra.run.smithi177.stdout:Install 36 Packages 2024-03-27T08:24:04.160 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:04.161 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 84 M 2024-03-27T08:24:04.161 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 921 M 2024-03-27T08:24:04.161 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:24:04.551 INFO:teuthology.orchestra.run.smithi177.stdout:(1/36): ceph-19.0.0-2463.g022fc18a.el9.x86_64.r 36 kB/s | 6.5 kB 00:00 2024-03-27T08:24:04.793 INFO:teuthology.orchestra.run.smithi177.stdout:(2/36): ceph-mds-19.0.0-2463.g022fc18a.el9.x86_ 5.2 MB/s | 2.2 MB 00:00 2024-03-27T08:24:04.835 INFO:teuthology.orchestra.run.smithi177.stdout:(3/36): ceph-mgr-19.0.0-2463.g022fc18a.el9.x86_ 3.4 MB/s | 1.6 MB 00:00 2024-03-27T08:24:04.935 INFO:teuthology.orchestra.run.smithi177.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2463.g022f 2.4 MB/s | 249 kB 00:00 2024-03-27T08:24:05.052 INFO:teuthology.orchestra.run.smithi177.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-03-27T08:24:05.094 INFO:teuthology.orchestra.run.smithi177.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-27T08:24:05.119 INFO:teuthology.orchestra.run.smithi177.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-27T08:24:05.145 INFO:teuthology.orchestra.run.smithi177.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.8 MB/s | 47 kB 00:00 2024-03-27T08:24:05.212 INFO:teuthology.orchestra.run.smithi177.stdout:(9/36): ceph-mon-19.0.0-2463.g022fc18a.el9.x86_ 7.6 MB/s | 5.0 MB 00:00 2024-03-27T08:24:05.229 INFO:teuthology.orchestra.run.smithi177.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 461 kB/s | 39 kB 00:00 2024-03-27T08:24:05.254 INFO:teuthology.orchestra.run.smithi177.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-27T08:24:05.312 INFO:teuthology.orchestra.run.smithi177.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-03-27T08:24:05.338 INFO:teuthology.orchestra.run.smithi177.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-27T08:24:05.363 INFO:teuthology.orchestra.run.smithi177.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-27T08:24:05.388 INFO:teuthology.orchestra.run.smithi177.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-27T08:24:05.464 INFO:teuthology.orchestra.run.smithi177.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-03-27T08:24:05.606 INFO:teuthology.orchestra.run.smithi177.stdout:(17/36): python3-requests-2.25.1-8.el9.noarch.r 516 kB/s | 125 kB 00:00 2024-03-27T08:24:05.756 INFO:teuthology.orchestra.run.smithi177.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 518 kB/s | 151 kB 00:00 2024-03-27T08:24:05.806 INFO:teuthology.orchestra.run.smithi177.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 217 kB/s | 43 kB 00:00 2024-03-27T08:24:05.873 INFO:teuthology.orchestra.run.smithi177.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-03-27T08:24:05.940 INFO:teuthology.orchestra.run.smithi177.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-03-27T08:24:06.007 INFO:teuthology.orchestra.run.smithi177.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 266 kB/s | 18 kB 00:00 2024-03-27T08:24:06.032 INFO:teuthology.orchestra.run.smithi177.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-03-27T08:24:06.083 INFO:teuthology.orchestra.run.smithi177.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-03-27T08:24:06.108 INFO:teuthology.orchestra.run.smithi177.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-03-27T08:24:06.158 INFO:teuthology.orchestra.run.smithi177.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 262 kB/s | 20 kB 00:00 2024-03-27T08:24:06.183 INFO:teuthology.orchestra.run.smithi177.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-03-27T08:24:06.234 INFO:teuthology.orchestra.run.smithi177.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-03-27T08:24:06.284 INFO:teuthology.orchestra.run.smithi177.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-27T08:24:06.309 INFO:teuthology.orchestra.run.smithi177.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 217 kB/s | 16 kB 00:00 2024-03-27T08:24:06.368 INFO:teuthology.orchestra.run.smithi177.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.1 MB/s | 90 kB 00:00 2024-03-27T08:24:06.386 INFO:teuthology.orchestra.run.smithi177.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-03-27T08:24:06.469 INFO:teuthology.orchestra.run.smithi177.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-27T08:24:06.502 INFO:teuthology.orchestra.run.smithi177.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-03-27T08:24:06.527 INFO:teuthology.orchestra.run.smithi177.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-03-27T08:24:07.986 INFO:teuthology.orchestra.run.smithi177.stdout:(36/36): ceph-osd-19.0.0-2463.g022fc18a.el9.x86 22 MB/s | 71 MB 00:03 2024-03-27T08:24:07.990 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:07.990 INFO:teuthology.orchestra.run.smithi177.stdout:Total 22 MB/s | 84 MB 00:03 2024-03-27T08:24:08.015 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:24:08.058 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:24:08.058 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:24:08.805 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:24:08.805 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:24:10.215 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:24:10.241 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:24:10.241 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:10.241 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T08:24:10.241 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:24:10.241 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:24:10.241 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:10.415 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:24:10.472 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-27T08:24:10.510 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-27T08:24:10.549 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-27T08:24:10.620 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-27T08:24:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-27T08:24:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-27T08:24:10.774 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:24:10.788 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-27T08:24:10.796 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:24:10.797 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:10.797 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T08:24:10.797 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:24:10.797 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:24:10.797 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:10.859 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-27T08:24:10.902 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-27T08:24:10.980 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-27T08:24:11.033 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:24:11.042 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-27T08:24:11.056 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:24:11.056 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:11.056 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T08:24:11.056 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:24:11.056 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:24:11.056 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:11.081 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-27T08:24:11.121 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:24:11.129 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-27T08:24:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-27T08:24:11.368 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-27T08:24:11.418 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-27T08:24:11.448 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-27T08:24:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-27T08:24:11.514 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-27T08:24:11.589 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-27T08:24:11.623 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:24:11.657 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-27T08:24:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-27T08:24:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-27T08:24:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-27T08:24:11.855 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-27T08:24:11.897 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-27T08:24:11.968 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-27T08:24:12.105 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-27T08:24:12.224 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:24:12.417 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 31/36 2024-03-27T08:24:12.444 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:24:12.469 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:24:12.469 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:12.469 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T08:24:12.469 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:24:12.469 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:24:12.469 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 1/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 3/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 4/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 5/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 6/36 2024-03-27T08:24:17.765 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-27T08:24:17.766 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-27T08:24:17.767 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-27T08:24:17.768 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T08:24:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T08:24:18.656 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T08:24:18.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T08:24:18.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T08:24:18.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T08:24:18.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T08:24:18.657 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:18.657 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:18.934 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-base 2024-03-27T08:24:19.525 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:27 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:19.626 INFO:teuthology.orchestra.run.smithi104.stdout:Package ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:24:19.680 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:19.681 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:24:19.681 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:19.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install cephadm 2024-03-27T08:24:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:28 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:20.463 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:20.464 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:20.464 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:20.464 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 762 k 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 762 k 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 768 k 2024-03-27T08:24:20.465 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:20.904 INFO:teuthology.orchestra.run.smithi104.stdout:cephadm-19.0.0-2463.g022fc18a.el9.noarch.rpm 1.7 MB/s | 762 kB 00:00 2024-03-27T08:24:20.904 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:20.905 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.7 MB/s | 762 kB 00:00 2024-03-27T08:24:20.905 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:20.910 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:20.910 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:20.915 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:20.916 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:20.986 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:21.383 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:21.458 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:21.856 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:22.122 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:22.123 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:22.123 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:22.123 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:22.123 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:22.123 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:22.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-immutable-object-cache 2024-03-27T08:24:22.888 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:30 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:23.038 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:23.039 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 157 k 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 157 k 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 640 k 2024-03-27T08:24:23.040 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:23.252 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:24:23.278 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:24:23.278 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:23.278 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T08:24:23.278 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:24:23.278 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:24:23.278 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:23.396 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-immutable-object-cache-19.0.0-2463.g022fc1 442 kB/s | 157 kB 00:00 2024-03-27T08:24:23.397 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:23.397 INFO:teuthology.orchestra.run.smithi104.stdout:Total 440 kB/s | 157 kB 00:00 2024-03-27T08:24:23.397 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:23.406 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:23.406 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:23.458 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:23.458 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:23.619 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:23.685 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:24:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:24:23.709 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:23.709 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T08:24:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:23.819 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:24:23.842 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:24:23.842 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:23.842 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T08:24:23.842 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:24:23.842 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:24:23.843 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:24:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:24:24.101 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:24.101 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T08:24:24.101 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:24:24.101 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:24:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:24.168 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:24:24.684 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:24:24.684 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:24.684 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:24.684 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:24.685 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:24.685 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:24.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr 2024-03-27T08:24:25.513 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:33 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:25.610 INFO:teuthology.orchestra.run.smithi104.stdout:Package ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:24:25.662 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:25.662 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:24:25.663 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:25.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-dashboard 2024-03-27T08:24:26.286 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:34 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:26.434 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:26.435 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-03-27T08:24:26.435 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:26.435 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 3.5 M 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 34 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 16 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T08:24:26.436 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout:Install 11 Packages 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 7.6 M 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 100 M 2024-03-27T08:24:26.437 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:27.062 INFO:teuthology.orchestra.run.smithi104.stdout:(1/11): ceph-grafana-dashboards-19.0.0-2463.g02 179 kB/s | 34 kB 00:00 2024-03-27T08:24:27.162 INFO:teuthology.orchestra.run.smithi104.stdout:(2/11): ceph-prometheus-alerts-19.0.0-2463.g022 57 kB/s | 16 kB 00:00 2024-03-27T08:24:27.187 INFO:teuthology.orchestra.run.smithi104.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 238 kB 00:00 2024-03-27T08:24:27.329 INFO:teuthology.orchestra.run.smithi104.stdout:(4/11): ceph-mgr-dashboard-19.0.0-2463.g022fc18 7.7 MB/s | 3.5 MB 00:00 2024-03-27T08:24:27.496 INFO:teuthology.orchestra.run.smithi104.stdout:(5/11): protobuf-compiler-3.14.0-13.el9.x86_64. 2.5 MB/s | 863 kB 00:00 2024-03-27T08:24:27.596 INFO:teuthology.orchestra.run.smithi104.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-27T08:24:27.938 INFO:teuthology.orchestra.run.smithi104.stdout:(7/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 730 kB/s | 548 kB 00:00 2024-03-27T08:24:28.021 INFO:teuthology.orchestra.run.smithi104.stdout:(8/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 340 kB/s | 144 kB 00:00 2024-03-27T08:24:28.213 INFO:teuthology.orchestra.run.smithi104.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 112 kB/s | 31 kB 00:00 2024-03-27T08:24:28.272 INFO:teuthology.orchestra.run.smithi104.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.6 MB/s | 2.0 MB 00:00 2024-03-27T08:24:28.530 INFO:teuthology.orchestra.run.smithi104.stdout:(11/11): python3-routes-2.5.1-5.el9.noarch.rpm 371 kB/s | 188 kB 00:00 2024-03-27T08:24:28.535 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:28.535 INFO:teuthology.orchestra.run.smithi104.stdout:Total 3.6 MB/s | 7.6 MB 00:02 2024-03-27T08:24:28.555 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:28.573 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:28.573 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:28.750 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:28.750 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:29.235 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:29.327 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-03-27T08:24:29.359 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-03-27T08:24:29.485 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-03-27T08:24:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-03-27T08:24:29.703 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-03-27T08:24:29.786 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-03-27T08:24:29.829 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-03-27T08:24:29.858 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:24:29.891 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 9/11 2024-03-27T08:24:30.833 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 1/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 3/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 4/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 5/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 6/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-27T08:24:30.834 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-27T08:24:30.835 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-27T08:24:30.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-27T08:24:30.939 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 10/11 2024-03-27T08:24:30.983 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T08:24:31.656 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T08:24:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T08:24:31.658 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T08:24:31.659 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:31.659 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:31.907 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-base 2024-03-27T08:24:32.217 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:24:32.217 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 1/11 2024-03-27T08:24:32.217 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 2/11 2024-03-27T08:24:32.217 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 3/11 2024-03-27T08:24:32.217 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-03-27T08:24:32.217 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-03-27T08:24:32.218 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-03-27T08:24:32.218 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-03-27T08:24:32.218 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:24:32.218 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-03-27T08:24:32.218 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-03-27T08:24:32.509 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:28 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:32.573 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-03-27T08:24:32.573 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:32.573 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:32.573 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:32.574 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:32.623 INFO:teuthology.orchestra.run.smithi177.stdout:Package ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:24:32.675 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:32.676 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:24:32.676 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:32.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install cephadm 2024-03-27T08:24:32.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-27T08:24:33.313 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:29 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:33.396 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:41 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:33.461 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:33.462 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:24:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:24:33.462 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:24:33.462 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 762 k 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 762 k 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 768 k 2024-03-27T08:24:33.463 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:24:33.546 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================ 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================ 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 7.4 M 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T08:24:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================ 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout:Install 12 Packages 2024-03-27T08:24:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:33.549 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 37 M 2024-03-27T08:24:33.549 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 209 M 2024-03-27T08:24:33.549 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:33.803 INFO:teuthology.orchestra.run.smithi104.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 180 kB/s | 32 kB 00:00 2024-03-27T08:24:33.837 INFO:teuthology.orchestra.run.smithi104.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 523 kB/s | 17 kB 00:00 2024-03-27T08:24:33.879 INFO:teuthology.orchestra.run.smithi104.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 863 kB/s | 34 kB 00:00 2024-03-27T08:24:33.929 INFO:teuthology.orchestra.run.smithi104.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.9 MB/s | 3.0 MB 00:00 2024-03-27T08:24:34.121 INFO:teuthology.orchestra.run.smithi104.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-03-27T08:24:34.188 INFO:teuthology.orchestra.run.smithi104.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-03-27T08:24:34.213 INFO:teuthology.orchestra.run.smithi104.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-27T08:24:34.310 INFO:teuthology.orchestra.run.smithi177.stdout:cephadm-19.0.0-2463.g022fc18a.el9.noarch.rpm 900 kB/s | 762 kB 00:00 2024-03-27T08:24:34.310 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:34.311 INFO:teuthology.orchestra.run.smithi177.stdout:Total 899 kB/s | 762 kB 00:00 2024-03-27T08:24:34.311 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:24:34.316 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:24:34.316 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:24:34.322 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:24:34.322 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:24:34.398 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:24:34.538 INFO:teuthology.orchestra.run.smithi104.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-03-27T08:24:34.697 INFO:teuthology.orchestra.run.smithi104.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-27T08:24:34.731 INFO:teuthology.orchestra.run.smithi104.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-03-27T08:24:34.849 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:34.939 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:35.332 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:35.464 INFO:teuthology.orchestra.run.smithi104.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 320 kB/s | 245 kB 00:00 2024-03-27T08:24:35.615 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:24:35.615 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:35.615 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:24:35.616 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:35.616 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:35.616 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:35.813 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-immutable-object-cache 2024-03-27T08:24:35.889 INFO:teuthology.orchestra.run.smithi104.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 3.3 MB/s | 7.4 MB 00:02 2024-03-27T08:24:35.894 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:35.894 INFO:teuthology.orchestra.run.smithi104.stdout:Total 16 MB/s | 37 MB 00:02 2024-03-27T08:24:35.936 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:35.961 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:35.961 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:36.328 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:36.328 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:36.385 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:32 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:36.535 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:36.536 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:24:36.536 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-03-27T08:24:36.536 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:24:36.536 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:24:36.537 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 157 k 2024-03-27T08:24:36.537 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:36.537 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:24:36.537 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:24:36.537 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:24:36.537 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:36.538 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 157 k 2024-03-27T08:24:36.538 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 640 k 2024-03-27T08:24:36.538 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:24:37.034 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:37.039 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-immutable-object-cache-19.0.0-2463.g022fc1 312 kB/s | 157 kB 00:00 2024-03-27T08:24:37.040 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:37.041 INFO:teuthology.orchestra.run.smithi177.stdout:Total 311 kB/s | 157 kB 00:00 2024-03-27T08:24:37.041 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:24:37.050 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:24:37.050 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:24:37.104 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:24:37.104 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:24:37.112 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-27T08:24:37.143 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-27T08:24:37.239 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:24:37.272 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:24:37.280 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-27T08:24:37.324 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:24:37.348 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:24:37.348 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:37.348 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T08:24:37.348 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:37.486 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:24:37.512 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:24:37.588 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-27T08:24:38.235 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-27T08:24:38.315 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:24:38.315 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:38.315 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:24:38.315 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:24:38.315 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:38.315 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:38.341 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-27T08:24:38.569 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr 2024-03-27T08:24:39.139 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:35 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:39.238 INFO:teuthology.orchestra.run.smithi177.stdout:Package ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:24:39.291 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:39.292 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:24:39.292 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:39.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-dashboard 2024-03-27T08:24:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-27T08:24:39.908 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:35 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:40.056 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================== 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================== 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 3.5 M 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 34 k 2024-03-27T08:24:40.057 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 16 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================== 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout:Install 11 Packages 2024-03-27T08:24:40.058 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:40.059 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 7.6 M 2024-03-27T08:24:40.059 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 100 M 2024-03-27T08:24:40.059 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:24:40.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-27T08:24:40.635 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:24:40.669 INFO:teuthology.orchestra.run.smithi177.stdout:(1/11): ceph-prometheus-alerts-19.0.0-2463.g022 39 kB/s | 16 kB 00:00 2024-03-27T08:24:40.744 INFO:teuthology.orchestra.run.smithi177.stdout:(2/11): ceph-grafana-dashboards-19.0.0-2463.g02 68 kB/s | 34 kB 00:00 2024-03-27T08:24:40.828 INFO:teuthology.orchestra.run.smithi177.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 238 kB 00:00 2024-03-27T08:24:40.961 INFO:teuthology.orchestra.run.smithi177.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 3.9 MB/s | 863 kB 00:00 2024-03-27T08:24:41.128 INFO:teuthology.orchestra.run.smithi177.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-03-27T08:24:41.212 INFO:teuthology.orchestra.run.smithi177.stdout:(6/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.4 MB/s | 548 kB 00:00 2024-03-27T08:24:41.287 INFO:teuthology.orchestra.run.smithi177.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-03-27T08:24:41.354 INFO:teuthology.orchestra.run.smithi177.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 461 kB/s | 31 kB 00:00 2024-03-27T08:24:41.438 INFO:teuthology.orchestra.run.smithi177.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-03-27T08:24:41.546 INFO:teuthology.orchestra.run.smithi177.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.9 MB/s | 2.0 MB 00:00 2024-03-27T08:24:41.663 INFO:teuthology.orchestra.run.smithi177.stdout:(11/11): ceph-mgr-dashboard-19.0.0-2463.g022fc1 2.5 MB/s | 3.5 MB 00:01 2024-03-27T08:24:41.666 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:41.666 INFO:teuthology.orchestra.run.smithi177.stdout:Total 4.7 MB/s | 7.6 MB 00:01 2024-03-27T08:24:41.684 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:24:41.702 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:24:41.702 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:24:41.904 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:24:41.904 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:24:42.379 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:24:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-03-27T08:24:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-03-27T08:24:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-03-27T08:24:42.733 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-03-27T08:24:42.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-03-27T08:24:42.869 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-03-27T08:24:42.913 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-03-27T08:24:42.942 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:24:42.974 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 9/11 2024-03-27T08:24:43.188 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:24:43.188 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 1/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-27T08:24:43.189 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-27T08:24:43.574 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-27T08:24:43.574 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:43.574 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T08:24:43.575 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T08:24:43.576 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:43.576 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:43.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-rook 2024-03-27T08:24:44.027 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 10/11 2024-03-27T08:24:44.066 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:24:44.474 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:52 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:44.622 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================== 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================== 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 49 k 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T08:24:44.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================== 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout:Install 13 Packages 2024-03-27T08:24:44.625 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:44.626 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 2.0 M 2024-03-27T08:24:44.626 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 26 M 2024-03-27T08:24:44.626 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:44.867 INFO:teuthology.orchestra.run.smithi104.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 184 kB/s | 18 kB 00:00 2024-03-27T08:24:44.917 INFO:teuthology.orchestra.run.smithi104.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 164 kB/s | 24 kB 00:00 2024-03-27T08:24:44.967 INFO:teuthology.orchestra.run.smithi104.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 174 kB 00:00 2024-03-27T08:24:44.993 INFO:teuthology.orchestra.run.smithi104.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-27T08:24:45.018 INFO:teuthology.orchestra.run.smithi104.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-27T08:24:45.043 INFO:teuthology.orchestra.run.smithi104.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 870 kB/s | 43 kB 00:00 2024-03-27T08:24:45.268 INFO:teuthology.orchestra.run.smithi104.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 129 kB/s | 32 kB 00:00 2024-03-27T08:24:45.294 INFO:teuthology.orchestra.run.smithi104.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 60 kB/s | 15 kB 00:00 2024-03-27T08:24:45.361 INFO:teuthology.orchestra.run.smithi104.stdout:(9/13): ceph-mgr-rook-19.0.0-2463.g022fc18a.el9 83 kB/s | 49 kB 00:00 2024-03-27T08:24:45.392 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:24:45.392 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 1/11 2024-03-27T08:24:45.392 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 2/11 2024-03-27T08:24:45.392 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 3/11 2024-03-27T08:24:45.392 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-03-27T08:24:45.393 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-03-27T08:24:45.393 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-03-27T08:24:45.393 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-03-27T08:24:45.393 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:24:45.393 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-03-27T08:24:45.393 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-03-27T08:24:45.486 INFO:teuthology.orchestra.run.smithi104.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 932 kB/s | 203 kB 00:00 2024-03-27T08:24:45.594 INFO:teuthology.orchestra.run.smithi104.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 828 kB/s | 90 kB 00:00 2024-03-27T08:24:45.620 INFO:teuthology.orchestra.run.smithi104.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 228 kB/s | 59 kB 00:00 2024-03-27T08:24:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T08:24:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T08:24:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T08:24:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:45.731 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:45.978 INFO:teuthology.orchestra.run.smithi104.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-03-27T08:24:45.980 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:45.980 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-03-27T08:24:45.981 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:45.984 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-27T08:24:45.995 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:45.995 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:46.090 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:46.092 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:46.537 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:42 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:46.604 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:46.686 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:46.687 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================ 2024-03-27T08:24:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================ 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 7.4 M 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T08:24:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================ 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout:Install 12 Packages 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:46.689 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 37 M 2024-03-27T08:24:46.690 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 209 M 2024-03-27T08:24:46.690 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:24:46.765 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-27T08:24:46.821 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-27T08:24:46.875 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-27T08:24:46.905 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-27T08:24:46.946 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-27T08:24:46.970 INFO:teuthology.orchestra.run.smithi177.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 264 kB/s | 32 kB 00:00 2024-03-27T08:24:46.995 INFO:teuthology.orchestra.run.smithi177.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 692 kB/s | 17 kB 00:00 2024-03-27T08:24:47.029 INFO:teuthology.orchestra.run.smithi177.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-27T08:24:47.038 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-27T08:24:47.142 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-27T08:24:47.190 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-27T08:24:47.196 INFO:teuthology.orchestra.run.smithi177.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 29 MB/s | 4.7 MB 00:00 2024-03-27T08:24:47.321 INFO:teuthology.orchestra.run.smithi177.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-27T08:24:47.363 INFO:teuthology.orchestra.run.smithi177.stdout:(6/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.2 MB/s | 217 kB 00:00 2024-03-27T08:24:47.463 INFO:teuthology.orchestra.run.smithi177.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.9 MB/s | 3.0 MB 00:00 2024-03-27T08:24:47.572 INFO:teuthology.orchestra.run.smithi177.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 7.4 MB/s | 803 kB 00:00 2024-03-27T08:24:47.722 INFO:teuthology.orchestra.run.smithi177.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-27T08:24:47.781 INFO:teuthology.orchestra.run.smithi177.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 945 kB/s | 195 kB 00:00 2024-03-27T08:24:47.896 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-27T08:24:47.944 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-27T08:24:47.978 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-27T08:24:47.981 INFO:teuthology.orchestra.run.smithi177.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 948 kB/s | 245 kB 00:00 2024-03-27T08:24:48.026 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-27T08:24:48.048 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:24:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:24:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 1/13 2024-03-27T08:24:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-27T08:24:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-27T08:24:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-27T08:24:48.855 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-27T08:24:48.856 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-27T08:24:48.973 INFO:teuthology.orchestra.run.smithi177.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 3.5 MB/s | 7.4 MB 00:02 2024-03-27T08:24:48.976 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:48.976 INFO:teuthology.orchestra.run.smithi177.stdout:Total 16 MB/s | 37 MB 00:02 2024-03-27T08:24:49.019 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:24:49.044 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:24:49.045 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:24:49.225 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T08:24:49.226 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T08:24:49.227 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T08:24:49.227 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T08:24:49.227 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T08:24:49.227 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T08:24:49.227 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:49.227 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:49.408 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:24:49.408 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:24:49.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-cephadm 2024-03-27T08:24:50.050 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:58 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:24:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-27T08:24:50.202 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-27T08:24:50.206 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:50.206 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================== 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================== 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 141 k 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:50.207 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:50.208 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================== 2024-03-27T08:24:50.208 INFO:teuthology.orchestra.run.smithi104.stdout:Install 6 Packages 2024-03-27T08:24:50.208 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:50.208 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 6.8 M 2024-03-27T08:24:50.208 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 33 M 2024-03-27T08:24:50.208 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:50.294 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:24:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-27T08:24:50.507 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:24:50.546 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:24:50.622 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-27T08:24:50.921 INFO:teuthology.orchestra.run.smithi104.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-27T08:24:51.105 INFO:teuthology.orchestra.run.smithi104.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2463.g022fc18a.e 301 kB/s | 141 kB 00:00 2024-03-27T08:24:51.230 INFO:teuthology.orchestra.run.smithi104.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2024-03-27T08:24:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-27T08:24:51.271 INFO:teuthology.orchestra.run.smithi104.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 346 kB/s | 58 kB 00:00 2024-03-27T08:24:51.297 INFO:teuthology.orchestra.run.smithi104.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 763 kB/s | 51 kB 00:00 2024-03-27T08:24:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-27T08:24:51.613 INFO:teuthology.orchestra.run.smithi104.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 253 kB/s | 248 kB 00:00 2024-03-27T08:24:51.615 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:51.616 INFO:teuthology.orchestra.run.smithi104.stdout:Total 4.8 MB/s | 6.8 MB 00:01 2024-03-27T08:24:51.628 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:51.638 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:51.638 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:51.726 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:51.727 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:52.019 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:52.188 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-27T08:24:52.232 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-27T08:24:52.604 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-27T08:24:52.745 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-27T08:24:52.842 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-27T08:24:52.905 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-27T08:24:52.935 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:24:53.624 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-27T08:24:53.653 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:24:53.788 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:24:53.788 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/6 2024-03-27T08:24:53.788 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-27T08:24:53.788 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-27T08:24:53.788 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-27T08:24:53.788 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-27T08:24:54.083 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-27T08:24:54.083 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:54.083 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:54.083 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:54.083 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T08:24:54.083 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T08:24:54.084 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T08:24:54.084 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T08:24:54.084 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T08:24:54.084 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:54.084 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:54.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-fuse 2024-03-27T08:24:54.903 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:02 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:55.052 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:55.052 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:55.052 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:55.052 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 870 k 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout:Install 2 Packages 2024-03-27T08:24:55.053 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:55.054 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 949 k 2024-03-27T08:24:55.054 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 4.0 M 2024-03-27T08:24:55.054 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:55.164 INFO:teuthology.orchestra.run.smithi104.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 713 kB/s | 79 kB 00:00 2024-03-27T08:24:55.981 INFO:teuthology.orchestra.run.smithi104.stdout:(2/2): ceph-fuse-19.0.0-2463.g022fc18a.el9.x86_ 939 kB/s | 870 kB 00:00 2024-03-27T08:24:55.982 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:55.983 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.0 MB/s | 949 kB 00:00 2024-03-27T08:24:55.983 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:56.005 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:56.005 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:56.078 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:56.079 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:56.265 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 1/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:24:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-27T08:24:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-27T08:24:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-27T08:24:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-27T08:24:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-27T08:24:56.416 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-27T08:24:56.489 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:24:56.699 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T08:24:56.700 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T08:24:56.701 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:56.701 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:24:57.141 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-rook 2024-03-27T08:24:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:24:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/2 2024-03-27T08:24:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-27T08:24:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:57.583 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:24:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-27T08:24:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:57.584 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:24:57.713 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:53 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:24:57.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-volume 2024-03-27T08:24:57.866 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================== 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================== 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 49 k 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:24:57.867 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:24:57.868 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================== 2024-03-27T08:24:57.869 INFO:teuthology.orchestra.run.smithi177.stdout:Install 13 Packages 2024-03-27T08:24:57.869 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:24:57.869 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 2.0 M 2024-03-27T08:24:57.869 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 26 M 2024-03-27T08:24:57.869 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:24:58.161 INFO:teuthology.orchestra.run.smithi177.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 237 kB/s | 24 kB 00:00 2024-03-27T08:24:58.212 INFO:teuthology.orchestra.run.smithi177.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 118 kB/s | 18 kB 00:00 2024-03-27T08:24:58.262 INFO:teuthology.orchestra.run.smithi177.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 174 kB 00:00 2024-03-27T08:24:58.295 INFO:teuthology.orchestra.run.smithi177.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-27T08:24:58.321 INFO:teuthology.orchestra.run.smithi177.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-03-27T08:24:58.363 INFO:teuthology.orchestra.run.smithi177.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 649 kB/s | 43 kB 00:00 2024-03-27T08:24:58.418 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:06 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:24:58.555 INFO:teuthology.orchestra.run.smithi177.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 78 kB/s | 15 kB 00:00 2024-03-27T08:24:58.569 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:24:58.570 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 264 k 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:Install 2 Packages 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 332 k 2024-03-27T08:24:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 1.5 M 2024-03-27T08:24:58.572 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:24:58.580 INFO:teuthology.orchestra.run.smithi177.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-03-27T08:24:58.680 INFO:teuthology.orchestra.run.smithi177.stdout:(9/13): ceph-mgr-rook-19.0.0-2463.g022fc18a.el9 79 kB/s | 49 kB 00:00 2024-03-27T08:24:58.702 INFO:teuthology.orchestra.run.smithi104.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 527 kB/s | 69 kB 00:00 2024-03-27T08:24:58.797 INFO:teuthology.orchestra.run.smithi177.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 837 kB/s | 203 kB 00:00 2024-03-27T08:24:58.889 INFO:teuthology.orchestra.run.smithi177.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 974 kB/s | 90 kB 00:00 2024-03-27T08:24:58.914 INFO:teuthology.orchestra.run.smithi177.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 252 kB/s | 59 kB 00:00 2024-03-27T08:24:58.948 INFO:teuthology.orchestra.run.smithi177.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.8 MB/s | 1.0 MB 00:00 2024-03-27T08:24:58.949 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:58.950 INFO:teuthology.orchestra.run.smithi177.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-03-27T08:24:58.950 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:24:58.965 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:24:58.965 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:24:59.063 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:24:59.063 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:24:59.218 INFO:teuthology.orchestra.run.smithi104.stdout:(2/2): ceph-volume-19.0.0-2463.g022fc18a.el9.no 408 kB/s | 264 kB 00:00 2024-03-27T08:24:59.220 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:24:59.220 INFO:teuthology.orchestra.run.smithi104.stdout:Total 512 kB/s | 332 kB 00:00 2024-03-27T08:24:59.220 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:24:59.229 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:24:59.229 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:24:59.270 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:24:59.270 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:24:59.492 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:24:59.579 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:24:59.668 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-27T08:24:59.705 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:24:59.720 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:24:59.720 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:24:59.720 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T08:24:59.720 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:24:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-27T08:24:59.807 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-27T08:24:59.862 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-27T08:24:59.900 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-27T08:24:59.941 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-27T08:25:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-27T08:25:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-27T08:25:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-27T08:25:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 1/2 2024-03-27T08:25:00.834 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-27T08:25:00.834 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:00.834 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:25:00.834 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:25:00.835 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T08:25:00.835 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:00.835 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-27T08:25:00.981 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-27T08:25:01.015 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-27T08:25:01.063 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-27T08:25:01.085 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:25:01.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librados-devel 2024-03-27T08:25:01.693 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:09 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:01.843 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 125 k 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:25:01.844 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:01.845 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 125 k 2024-03-27T08:25:01.845 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 445 k 2024-03-27T08:25:01.845 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:25:01.916 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:25:01.916 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 1/13 2024-03-27T08:25:01.916 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-27T08:25:01.916 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-27T08:25:01.916 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-27T08:25:01.917 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T08:25:02.288 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:02.289 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:02.523 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-cephadm 2024-03-27T08:25:02.632 INFO:teuthology.orchestra.run.smithi104.stdout:librados-devel-19.0.0-2463.g022fc18a.el9.x86_64 159 kB/s | 125 kB 00:00 2024-03-27T08:25:02.633 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:02.633 INFO:teuthology.orchestra.run.smithi104.stdout:Total 158 kB/s | 125 kB 00:00 2024-03-27T08:25:02.633 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:25:02.639 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:25:02.639 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:25:02.697 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:25:02.697 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:25:02.837 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:25:02.947 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:03.090 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:59 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:03.241 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:03.242 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================== 2024-03-27T08:25:03.242 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:25:03.242 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================== 2024-03-27T08:25:03.242 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:03.242 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 141 k 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================== 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout:Install 6 Packages 2024-03-27T08:25:03.243 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:03.244 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 6.8 M 2024-03-27T08:25:03.244 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 33 M 2024-03-27T08:25:03.244 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:03.381 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:03.665 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:03.665 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:03.666 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:25:03.666 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:03.666 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:03.666 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:03.842 INFO:teuthology.orchestra.run.smithi177.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-03-27T08:25:03.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install libcephfs2 2024-03-27T08:25:04.465 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:12 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:04.483 INFO:teuthology.orchestra.run.smithi177.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 6.7 MB/s | 5.8 MB 00:00 2024-03-27T08:25:04.564 INFO:teuthology.orchestra.run.smithi104.stdout:Package libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:04.616 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:04.617 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:04.617 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:04.689 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install libcephfs-devel 2024-03-27T08:25:05.017 INFO:teuthology.orchestra.run.smithi177.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 467 kB/s | 548 kB 00:01 2024-03-27T08:25:05.068 INFO:teuthology.orchestra.run.smithi177.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 99 kB/s | 58 kB 00:00 2024-03-27T08:25:05.151 INFO:teuthology.orchestra.run.smithi177.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 382 kB/s | 51 kB 00:00 2024-03-27T08:25:05.245 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:13 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:05.393 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:05.393 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:05.393 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 32 k 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 32 k 2024-03-27T08:25:05.394 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 142 k 2024-03-27T08:25:05.395 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:25:05.401 INFO:teuthology.orchestra.run.smithi177.stdout:(6/6): ceph-mgr-cephadm-19.0.0-2463.g022fc18a.e 79 kB/s | 141 kB 00:01 2024-03-27T08:25:05.403 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:05.403 INFO:teuthology.orchestra.run.smithi177.stdout:Total 3.1 MB/s | 6.8 MB 00:02 2024-03-27T08:25:05.418 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:05.430 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:05.430 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:05.522 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:05.523 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:05.819 INFO:teuthology.orchestra.run.smithi104.stdout:libcephfs-devel-19.0.0-2463.g022fc18a.el9.x86_6 74 kB/s | 32 kB 00:00 2024-03-27T08:25:05.820 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:05.820 INFO:teuthology.orchestra.run.smithi104.stdout:Total 74 kB/s | 32 kB 00:00 2024-03-27T08:25:05.820 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:25:05.824 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:25:05.824 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:25:05.825 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:05.847 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:25:05.848 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:25:05.926 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:25:06.000 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-27T08:25:06.051 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:06.052 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-27T08:25:06.433 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:06.570 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-27T08:25:06.663 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-27T08:25:06.697 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:06.697 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:06.697 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:25:06.697 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:06.698 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:06.698 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:06.726 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-27T08:25:06.748 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:25:06.878 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librados2 2024-03-27T08:25:07.434 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:15 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:07.529 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:07.559 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:25:07.559 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/6 2024-03-27T08:25:07.559 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-27T08:25:07.559 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-27T08:25:07.559 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-27T08:25:07.559 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-27T08:25:07.580 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:07.582 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:07.582 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:07.654 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librbd1 2024-03-27T08:25:07.867 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-27T08:25:07.867 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:07.867 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:07.867 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:07.868 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:08.106 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-fuse 2024-03-27T08:25:08.205 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:16 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:08.301 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:08.352 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:08.353 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:08.353 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:08.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rados 2024-03-27T08:25:08.670 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:04 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:08.821 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 870 k 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:08.822 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:08.823 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:08.823 INFO:teuthology.orchestra.run.smithi177.stdout:Install 2 Packages 2024-03-27T08:25:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:08.823 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 949 k 2024-03-27T08:25:08.823 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 4.0 M 2024-03-27T08:25:08.823 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:08.961 INFO:teuthology.orchestra.run.smithi177.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 571 kB/s | 79 kB 00:00 2024-03-27T08:25:08.975 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:16 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:09.071 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:09.123 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:09.124 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:09.124 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:09.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rgw 2024-03-27T08:25:09.753 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:17 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:09.849 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:09.901 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:09.902 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:09.902 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:09.919 INFO:teuthology.orchestra.run.smithi177.stdout:(2/2): ceph-fuse-19.0.0-2463.g022fc18a.el9.x86_ 794 kB/s | 870 kB 00:01 2024-03-27T08:25:09.921 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:09.921 INFO:teuthology.orchestra.run.smithi177.stdout:Total 864 kB/s | 949 kB 00:01 2024-03-27T08:25:09.921 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:09.944 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:09.944 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:09.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-cephfs 2024-03-27T08:25:10.017 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:10.018 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:10.221 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:10.362 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-27T08:25:10.436 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:25:10.534 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:18 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:10.630 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:10.681 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:10.682 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:10.682 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:10.753 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rbd 2024-03-27T08:25:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:25:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/2 2024-03-27T08:25:11.307 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:19 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:11.403 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:11.447 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-27T08:25:11.448 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:11.448 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:11.448 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-27T08:25:11.448 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:11.448 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:11.455 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:11.456 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-03-27T08:25:11.456 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:11.518 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-fuse 2024-03-27T08:25:11.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-volume 2024-03-27T08:25:12.072 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:20 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:12.219 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:12.220 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-03-27T08:25:12.220 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 90 k 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 90 k 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 313 k 2024-03-27T08:25:12.221 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:25:12.289 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:08 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 264 k 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T08:25:12.439 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T08:25:12.440 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:12.440 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:12.440 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:12.440 INFO:teuthology.orchestra.run.smithi177.stdout:Install 2 Packages 2024-03-27T08:25:12.440 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:12.440 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 332 k 2024-03-27T08:25:12.441 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 1.5 M 2024-03-27T08:25:12.441 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:12.577 INFO:teuthology.orchestra.run.smithi177.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 507 kB/s | 69 kB 00:00 2024-03-27T08:25:12.905 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-fuse-19.0.0-2463.g022fc18a.el9.x86_64.rpm 131 kB/s | 90 kB 00:00 2024-03-27T08:25:12.906 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:12.906 INFO:teuthology.orchestra.run.smithi104.stdout:Total 131 kB/s | 90 kB 00:00 2024-03-27T08:25:12.906 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:25:12.915 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:25:12.915 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:25:12.972 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:25:12.972 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:25:13.082 INFO:teuthology.orchestra.run.smithi177.stdout:(2/2): ceph-volume-19.0.0-2463.g022fc18a.el9.no 411 kB/s | 264 kB 00:00 2024-03-27T08:25:13.084 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:13.084 INFO:teuthology.orchestra.run.smithi177.stdout:Total 516 kB/s | 332 kB 00:00 2024-03-27T08:25:13.084 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:13.093 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:13.093 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:13.112 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:25:13.135 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:13.136 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:13.204 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:13.336 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:13.516 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-27T08:25:13.551 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:25:13.567 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:25:13.567 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:25:13.567 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T08:25:13.567 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:13.595 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:13.877 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:13.878 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:13.878 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:25:13.878 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:13.878 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:13.878 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:14.113 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-mirror 2024-03-27T08:25:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 1/2 2024-03-27T08:25:14.671 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-27T08:25:14.671 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:14.671 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:14.671 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:25:14.672 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T08:25:14.672 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:14.672 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:14.676 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:22 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:14.827 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 3.4 M 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 29 M 2024-03-27T08:25:14.828 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:25:14.965 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados-devel 2024-03-27T08:25:15.540 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:11 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:15.688 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:15.689 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:15.689 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-03-27T08:25:15.689 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:15.689 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 125 k 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 125 k 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 445 k 2024-03-27T08:25:15.690 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:16.479 INFO:teuthology.orchestra.run.smithi177.stdout:librados-devel-19.0.0-2463.g022fc18a.el9.x86_64 158 kB/s | 125 kB 00:00 2024-03-27T08:25:16.480 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:16.480 INFO:teuthology.orchestra.run.smithi177.stdout:Total 158 kB/s | 125 kB 00:00 2024-03-27T08:25:16.481 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:16.486 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:16.486 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:16.544 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:16.544 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:16.692 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:16.795 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:17.141 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-mirror-19.0.0-2463.g022fc18a.el9.x86_64.rpm 1.5 MB/s | 3.4 MB 00:02 2024-03-27T08:25:17.142 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:17.142 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.5 MB/s | 3.4 MB 00:02 2024-03-27T08:25:17.143 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:25:17.155 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:25:17.155 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:25:17.233 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:25:17.234 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:25:17.283 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:17.565 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:17.565 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:17.565 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:17.565 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:17.566 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:17.566 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:17.798 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:25:17.818 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install libcephfs2 2024-03-27T08:25:17.850 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:17.873 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:17.873 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:25:17.874 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T08:25:17.874 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:25:17.874 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:25:17.874 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:18.392 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:14 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:18.492 INFO:teuthology.orchestra.run.smithi177.stdout:Package libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:18.545 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:18.546 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:18.546 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:18.606 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install libcephfs-devel 2024-03-27T08:25:19.112 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:19.112 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:19.112 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:25:19.112 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:19.112 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:19.112 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:19.171 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:15 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:19.320 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 32 k 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:25:19.322 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:19.322 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 32 k 2024-03-27T08:25:19.322 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 142 k 2024-03-27T08:25:19.322 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:19.370 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-nbd 2024-03-27T08:25:19.785 INFO:teuthology.orchestra.run.smithi177.stdout:libcephfs-devel-19.0.0-2463.g022fc18a.el9.x86_6 68 kB/s | 32 kB 00:00 2024-03-27T08:25:19.785 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:19.785 INFO:teuthology.orchestra.run.smithi177.stdout:Total 68 kB/s | 32 kB 00:00 2024-03-27T08:25:19.786 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:19.789 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:19.789 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:19.915 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:19.943 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:02:27 ago on Wed 27 Mar 2024 08:22:52 AM UTC. 2024-03-27T08:25:20.033 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:20.090 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-03-27T08:25:20.091 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:20.091 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-03-27T08:25:20.091 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:20.091 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-03-27T08:25:20.091 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 175 k 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 175 k 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 673 k 2024-03-27T08:25:20.092 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-03-27T08:25:20.462 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:20.715 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:20.715 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:20.715 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:20.715 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:20.715 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:20.715 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:20.884 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados2 2024-03-27T08:25:21.128 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-nbd-19.0.0-2463.g022fc18a.el9.x86_64.rpm 169 kB/s | 175 kB 00:01 2024-03-27T08:25:21.129 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:21.129 INFO:teuthology.orchestra.run.smithi104.stdout:Total 169 kB/s | 175 kB 00:01 2024-03-27T08:25:21.130 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-03-27T08:25:21.146 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-03-27T08:25:21.146 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-03-27T08:25:21.208 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-03-27T08:25:21.208 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-03-27T08:25:21.360 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-03-27T08:25:21.443 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:17 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:21.470 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:21.539 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:21.592 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:21.593 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:21.593 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:21.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librbd1 2024-03-27T08:25:21.944 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:22.213 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:18 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:22.252 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:22.252 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:22.252 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-03-27T08:25:22.252 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:22.253 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:22.253 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-03-27T08:25:22.311 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:22.365 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:22.366 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:22.366 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:22.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rados 2024-03-27T08:25:22.490 DEBUG:teuthology.parallel:result is None 2024-03-27T08:25:22.987 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:18 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:23.084 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:23.136 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:23.137 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:23.137 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:23.196 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rgw 2024-03-27T08:25:23.756 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:19 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:23.853 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:23.905 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:23.906 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:23.906 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:23.966 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-cephfs 2024-03-27T08:25:24.525 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:20 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:24.622 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:24.674 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:24.675 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:24.675 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:24.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rbd 2024-03-27T08:25:25.290 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:21 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:25.386 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:25:25.439 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:25.440 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-03-27T08:25:25.440 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:25.500 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-fuse 2024-03-27T08:25:26.062 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:22 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:26.210 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:26.211 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:26.211 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-03-27T08:25:26.211 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 90 k 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 90 k 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 313 k 2024-03-27T08:25:26.212 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:26.679 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-fuse-19.0.0-2463.g022fc18a.el9.x86_64.rpm 192 kB/s | 90 kB 00:00 2024-03-27T08:25:26.680 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:26.680 INFO:teuthology.orchestra.run.smithi177.stdout:Total 192 kB/s | 90 kB 00:00 2024-03-27T08:25:26.680 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:26.689 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:26.689 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:26.744 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:26.744 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:26.885 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:26.994 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:27.705 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:27.705 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:27.970 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-mirror 2024-03-27T08:25:28.534 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:24 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:28.692 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 3.4 M 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 29 M 2024-03-27T08:25:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:29.807 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-mirror-19.0.0-2463.g022fc18a.el9.x86_64.rpm 3.1 MB/s | 3.4 MB 00:01 2024-03-27T08:25:29.807 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:29.808 INFO:teuthology.orchestra.run.smithi177.stdout:Total 3.0 MB/s | 3.4 MB 00:01 2024-03-27T08:25:29.808 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:29.820 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:29.820 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:29.897 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:29.897 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:30.461 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:30.533 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:30.557 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:30.558 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:25:30.558 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T08:25:30.558 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:25:30.558 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:25:30.558 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:31.843 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:31.843 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:32.100 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-nbd 2024-03-27T08:25:32.699 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:02:28 ago on Wed 27 Mar 2024 08:23:04 AM UTC. 2024-03-27T08:25:32.853 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 175 k 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 175 k 2024-03-27T08:25:32.854 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 673 k 2024-03-27T08:25:32.855 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T08:25:33.636 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-nbd-19.0.0-2463.g022fc18a.el9.x86_64.rpm 224 kB/s | 175 kB 00:00 2024-03-27T08:25:33.637 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:25:33.637 INFO:teuthology.orchestra.run.smithi177.stdout:Total 224 kB/s | 175 kB 00:00 2024-03-27T08:25:33.637 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T08:25:33.650 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T08:25:33.650 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T08:25:33.704 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T08:25:33.704 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T08:25:33.854 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T08:25:33.959 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:34.451 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:34.737 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:25:34.737 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:34.737 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T08:25:34.737 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:25:34.737 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:34.737 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T08:25:34.960 DEBUG:teuthology.parallel:result is None 2024-03-27T08:25:34.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:25:35.111 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T08:25:35.137 INFO:teuthology.orchestra.run.smithi104.stdout:19.0.0-2463.g022fc18a.el9 2024-03-27T08:25:35.137 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2463.g022fc18a.el9 2024-03-27T08:25:35.138 INFO:teuthology.task.install:The correct ceph version 19.0.0-2463.g022fc18a is installed. 2024-03-27T08:25:35.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:25:35.286 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T08:25:35.308 INFO:teuthology.orchestra.run.smithi177.stdout:19.0.0-2463.g022fc18a.el9 2024-03-27T08:25:35.308 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2463.g022fc18a.el9 2024-03-27T08:25:35.308 INFO:teuthology.task.install:The correct ceph version 19.0.0-2463.g022fc18a is installed. 2024-03-27T08:25:35.310 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-27T08:25:35.310 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:35.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T08:25:35.342 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:35.342 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T08:25:35.377 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-27T08:25:35.378 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:35.378 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T08:25:35.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T08:25:35.484 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:35.484 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T08:25:35.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T08:25:35.579 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-27T08:25:35.579 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:35.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T08:25:35.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T08:25:35.679 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:35.680 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T08:25:35.707 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T08:25:35.774 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-27T08:25:35.774 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:35.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T08:25:35.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T08:25:35.873 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:35.873 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T08:25:35.900 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T08:25:35.968 INFO:teuthology.run_tasks:Running task ceph... 2024-03-27T08:25:36.076 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-27T08:25:36.093 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /var/log/ceph 2024-03-27T08:25:36.096 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2024-03-27T08:25:36.127 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-27T08:25:36.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-27T08:25:36.171 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-27T08:25:36.199 INFO:tasks.ceph:Creating extra log directories... 2024-03-27T08:25:36.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-27T08:25:36.242 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-27T08:25:36.297 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-27T08:25:36.297 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-27T08:25:36.298 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881', 'branch': 'wip-matanb-crimson-foreign-ptr', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7625881', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-foreign-ptr'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'eae914f097640d21427554e7f6aa6f1e9c4a7748'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 7338, 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'eae914f097640d21427554e7f6aa6f1e9c4a7748', 'targets': {'smithi104.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVyO7I+u8QQoEIR1UvSZVM4yfR+p9m5dh/TewU1IX0dTmvsTWenBaVLx8NHfIV/mDD0qBDszdiLSUURv4ZfSfc=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+RDlkkGXbTGmZGsjfG3RBy0+cKR5y1laHPk2EWs5LpMJ+c0Do2UyQm7JiByO/HcdIeY5HRPYduZtEUnASxpn0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-27_08:05:55', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-27T08:25:36.298 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-27T08:25:36.311 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-27T08:25:36.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/run/ceph 2024-03-27T08:25:36.367 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2024-03-27T08:25:36.395 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:36.395 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T08:25:36.451 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T08:25:36.452 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-03-27 08:25:18.131299249 +0000 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-03-27 08:22:07.513893410 +0000 2024-03-27T08:25:36.508 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-03-27 08:22:07.513893410 +0000 2024-03-27T08:25:36.509 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-03-27 08:22:07.513893410 +0000 2024-03-27T08:25:36.509 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T08:25:36.575 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-03-27T08:25:36.575 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-03-27T08:25:36.575 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000247492 s, 2.1 MB/s 2024-03-27T08:25:36.576 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T08:25:36.633 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-03-27 08:25:18.132299238 +0000 2024-03-27T08:25:36.689 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-03-27 08:22:07.869889222 +0000 2024-03-27T08:25:36.690 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-03-27 08:22:07.869889222 +0000 2024-03-27T08:25:36.690 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-03-27 08:22:07.869889222 +0000 2024-03-27T08:25:36.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T08:25:36.755 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-03-27T08:25:36.755 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-03-27T08:25:36.756 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000184675 s, 2.8 MB/s 2024-03-27T08:25:36.756 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T08:25:36.813 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-03-27T08:25:36.868 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T08:25:36.868 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:36.868 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-27T08:25:36.869 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:36.869 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:36.869 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-03-27 08:25:18.132299238 +0000 2024-03-27T08:25:36.869 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-03-27 08:22:08.213885175 +0000 2024-03-27T08:25:36.869 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-03-27 08:22:08.213885175 +0000 2024-03-27T08:25:36.869 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-03-27 08:22:08.213885175 +0000 2024-03-27T08:25:36.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T08:25:36.933 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-03-27T08:25:36.933 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-03-27T08:25:36.933 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000197417 s, 2.6 MB/s 2024-03-27T08:25:36.935 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T08:25:36.991 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-03-27T08:25:37.045 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-03-27 08:25:18.132299238 +0000 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-03-27 08:22:08.545881268 +0000 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-03-27 08:22:08.545881268 +0000 2024-03-27T08:25:37.046 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-03-27 08:22:08.545881268 +0000 2024-03-27T08:25:37.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T08:25:37.111 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-03-27T08:25:37.111 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-03-27T08:25:37.111 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000172945 s, 3.0 MB/s 2024-03-27T08:25:37.112 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T08:25:37.167 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-27T08:25:37.168 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:37.168 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T08:25:37.185 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T08:25:37.185 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-03-27T08:25:37.242 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T08:25:37.242 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:37.242 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-27T08:25:37.242 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:37.242 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:37.243 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 08:25:30.812224661 +0000 2024-03-27T08:25:37.243 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 08:21:58.965271697 +0000 2024-03-27T08:25:37.243 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 08:21:58.965271697 +0000 2024-03-27T08:25:37.243 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 08:21:58.965271697 +0000 2024-03-27T08:25:37.243 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T08:25:37.310 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T08:25:37.310 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T08:25:37.310 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000210952 s, 2.4 MB/s 2024-03-27T08:25:37.311 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T08:25:37.368 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-03-27T08:25:37.425 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T08:25:37.425 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:37.425 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-27T08:25:37.425 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:37.426 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:37.426 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 08:25:30.813224634 +0000 2024-03-27T08:25:37.426 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 08:21:59.290263042 +0000 2024-03-27T08:25:37.426 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 08:21:59.290263042 +0000 2024-03-27T08:25:37.426 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 08:21:59.290263042 +0000 2024-03-27T08:25:37.426 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T08:25:37.493 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T08:25:37.493 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T08:25:37.493 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000200535 s, 2.6 MB/s 2024-03-27T08:25:37.494 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T08:25:37.552 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 08:25:30.813224634 +0000 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 08:21:59.620254255 +0000 2024-03-27T08:25:37.608 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 08:21:59.620254255 +0000 2024-03-27T08:25:37.609 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 08:21:59.620254255 +0000 2024-03-27T08:25:37.609 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T08:25:37.675 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T08:25:37.675 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T08:25:37.675 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000186378 s, 2.7 MB/s 2024-03-27T08:25:37.676 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T08:25:37.732 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 08:25:30.814224607 +0000 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 08:21:59.956245308 +0000 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 08:21:59.956245308 +0000 2024-03-27T08:25:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 08:21:59.956245308 +0000 2024-03-27T08:25:37.790 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T08:25:37.856 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T08:25:37.856 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T08:25:37.856 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000187912 s, 2.7 MB/s 2024-03-27T08:25:37.857 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T08:25:37.913 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-27T08:25:37.913 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-03-27T08:25:37.913 INFO:tasks.ceph:Generating config... 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-03-27T08:25:37.915 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-03-27T08:25:37.915 INFO:tasks.ceph:[osd] debug monc = 20 2024-03-27T08:25:37.915 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-03-27T08:25:37.915 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-03-27T08:25:37.916 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-27T08:25:37.916 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-27T08:25:37.916 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-03-27T08:25:37.916 INFO:tasks.ceph:[osd] osd class default list = * 2024-03-27T08:25:37.916 INFO:tasks.ceph:[osd] osd class load list = * 2024-03-27T08:25:37.916 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-03-27T08:25:37.916 INFO:tasks.ceph:[client] debug ms = 1 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] ms client mode = crc 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-03-27T08:25:37.916 INFO:tasks.ceph:[global] ms service mode = crc 2024-03-27T08:25:37.916 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-27T08:25:37.917 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-27T08:25:37.917 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-03-27T08:25:37.917 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-03-27T08:25:37.917 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-03-27T08:25:37.917 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-03-27T08:25:37.917 INFO:tasks.ceph:Setting up mon.a... 2024-03-27T08:25:37.917 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-27T08:25:37.980 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.keyring 2024-03-27T08:25:37.983 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-27T08:25:38.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-27T08:25:38.064 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.104')] 2024-03-27T08:25:38.064 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.104', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-03-27T08:25:38.065 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:38.065 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-27T08:25:38.119 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.104 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:38.207 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:25:38.215+0000 7fdb5e628f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:25:38.208 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:25:38.215+0000 7fdb5e628f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:25:38.208 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:25:38.215+0000 7fdb5e628f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:25:38.208 INFO:teuthology.orchestra.run.smithi104.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:38.208 INFO:teuthology.orchestra.run.smithi104.stdout:monmaptool: generated fsid 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:setting min_mon_release = quincy 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:epoch 0 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:fsid 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:last_changed 2024-03-27T08:25:38.216819+0000 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:created 2024-03-27T08:25:38.216819+0000 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:min_mon_release 17 (quincy) 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:election_strategy: 1 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:0: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.a 2024-03-27T08:25:38.209 INFO:teuthology.orchestra.run.smithi104.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-03-27T08:25:38.210 DEBUG:teuthology.orchestra.run.smithi104:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-27T08:25:38.225 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 756f8b0d-eb2f-4187-9610-708a3d134344... 2024-03-27T08:25:38.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-27T08:25:38.267 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout:[global] 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: chdir = "" 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: auth supported = cephx 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: filestore xattr use omap = true 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: mon clock drift allowed = 1.000 2024-03-27T08:25:38.315 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: osd crush chooseleaf type = 0 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: auth debug = true 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on old message = true 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on bug = true 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: mon pg warn max object skew = 0 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: # disable pg_autoscaler by default for new pools 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default size = 2 2024-03-27T08:25:38.316 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow primary affinity = true 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow pg remap = true 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on legacy crush tunables = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on crush straw calc version zero = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on no sortbitwise = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on osd down out interval zero = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on too few osds = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: mon_allow_pool_size_one = true 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.317 INFO:teuthology.orchestra.run.smithi104.stdout: osd default data pool replay window = 5 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: mon allow pool delete = true 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: mon cluster log file level = debug 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: debug asserts on shutdown = true 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: mon health detail to clog = false 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: mon host = 172.21.15.104 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: ms client mode = crc 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: ms cluster mode = crc 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: ms mon client mode = crc 2024-03-27T08:25:38.318 INFO:teuthology.orchestra.run.smithi104.stdout: ms mon cluster mode = crc 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: ms mon service mode = crc 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: ms service mode = crc 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: fsid = 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout:[osd] 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: osd journal size = 100 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub load threshold = 5.0 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub max interval = 600 2024-03-27T08:25:38.319 INFO:teuthology.orchestra.run.smithi104.stdout: osd mclock profile = high_recovery_ops 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd recover clone overlap = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd recovery max chunk = 1048576 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug shutdown = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug op order = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug verify stray on activate = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug trim objects = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd open classes on start = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug pg log writeout = true 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: osd deep scrub update digest min age = 30 2024-03-27T08:25:38.320 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: osd map max advance = 10 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: journal zero on create = true 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: filestore ondisk finisher threads = 3 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: filestore apply finisher threads = 3 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: bdev debug aio = true 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug misdirected ops = true 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: debug monc = 20 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: crimson alien thread cpu cores = 6-7 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: crimson osd obc lru size = 10 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-03-27T08:25:38.321 INFO:teuthology.orchestra.run.smithi104.stdout: debug osd = 20 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: osd blocked scrub grace period = 3600 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: osd class default list = * 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: osd class load list = * 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: osd shutdown pgref assert = True 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr] 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: debug mgr = 20 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: mgr/telemetry/nag = false 2024-03-27T08:25:38.322 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout:[mon] 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: debug paxos = 20 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: mon data avail warn = 5 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: mon mgr mkfs grace = 240 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd reporter subtree level = osd 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd prime pg temp = true 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-27T08:25:38.323 INFO:teuthology.orchestra.run.smithi104.stdout: auth mon ticket ttl = 660 # 11m 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: auth service ticket ttl = 240 # 4m 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: # don't complain about insecure global_id in the test suite 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: # 1m isn't quite enough 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon_down_mkfs_grace = 2m 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_filestore_osds = false 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon min osdmap epochs = 50 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon osdmap full prune interval = 2 2024-03-27T08:25:38.324 INFO:teuthology.orchestra.run.smithi104.stdout: mon osdmap full prune min = 15 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: mon osdmap full prune txsize = 2 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: paxos service trim min = 10 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on pool no app = False 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default crimson = True 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout:[client] 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: rgw cache enabled = true 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable ops log = true 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable usage log = true 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.a] 2024-03-27T08:25:38.325 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.0] 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.1] 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.2] 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.3] 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi104.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:25:38.326 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-03-27T08:25:38.327 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-03-27T08:25:38.328 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-03-27T08:25:38.329 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = 172.21.15.104 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: ms client mode = crc 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: ms cluster mode = crc 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: ms mon client mode = crc 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: ms mon cluster mode = crc 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: ms mon service mode = crc 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: ms service mode = crc 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.330 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-03-27T08:25:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: debug monc = 20 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: crimson alien thread cpu cores = 6-7 2024-03-27T08:25:38.332 INFO:teuthology.orchestra.run.smithi177.stdout: crimson osd obc lru size = 10 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: osd blocked scrub grace period = 3600 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: osd class default list = * 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: osd class load list = * 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: osd shutdown pgref assert = True 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:25:38.333 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:25:38.334 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: mon min osdmap epochs = 50 2024-03-27T08:25:38.335 INFO:teuthology.orchestra.run.smithi177.stdout: mon osdmap full prune interval = 2 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: mon osdmap full prune min = 15 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: mon osdmap full prune txsize = 2 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: paxos service trim min = 10 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on pool no app = False 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default crimson = True 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-03-27T08:25:38.336 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2024-03-27T08:25:38.337 INFO:teuthology.orchestra.run.smithi177.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:25:38.339 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-27T08:25:38.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-27T08:25:38.445 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-27T08:25:38.445 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:38.445 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-27T08:25:38.460 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:38.460 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-27T08:25:38.515 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:25:38.515 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:38.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-27T08:25:38.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-27T08:25:38.594 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:25:38.594 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:38.649 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:25:38.649 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:38.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-27T08:25:38.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-27T08:25:38.690 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:25:38.690 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:38.746 INFO:tasks.ceph:Setting up mon nodes... 2024-03-27T08:25:38.746 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-27T08:25:38.746 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-27T08:25:38.891 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-27T08:25:38.894 INFO:tasks.ceph:Setting up mds nodes... 2024-03-27T08:25:38.894 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-27T08:25:38.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-27T08:25:38.946 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-27T08:25:38.962 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-27T08:25:39.013 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-27T08:25:39.029 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-27T08:25:39.029 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-03-27T08:25:39.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-27T08:25:39.056 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-27T08:25:39.056 INFO:tasks.ceph:role: osd.0 2024-03-27T08:25:39.056 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:25:39.056 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-27T08:25:39.130 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:25:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:25:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:25:39.131 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:25:39.641 INFO:teuthology.orchestra.run.smithi104.stdout:Discarding blocks...Done. 2024-03-27T08:25:39.643 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-03-27T08:25:39.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-27T08:25:39.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-27T08:25:39.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-27T08:25:39.885 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-27T08:25:39.885 INFO:tasks.ceph:role: osd.1 2024-03-27T08:25:39.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:25:39.885 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-27T08:25:39.959 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:25:39.959 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:25:39.959 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:25:39.959 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:25:39.959 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:25:39.959 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:25:39.960 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:25:39.960 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:25:39.960 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:25:39.960 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:25:40.414 INFO:teuthology.orchestra.run.smithi104.stdout:Discarding blocks...Done. 2024-03-27T08:25:40.416 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-03-27T08:25:40.416 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-27T08:25:40.510 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-27T08:25:40.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:40.923 INFO:teuthology.orchestra.run.smithi104.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:40.924 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:40,933 seastar - Reactor backend: linux-aio 2024-03-27T08:25:40.946 INFO:teuthology.orchestra.run.smithi104.stderr:==29217==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:41.183 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:41,191 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:25:41.183 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:41,191 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-03-27T08:25:41.184 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG 2024-03-27 08:25:41,193 [shard 0:main] seastar - reactor::drain 2024-03-27T08:25:41.765 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:41.765 INFO:teuthology.orchestra.run.smithi104.stderr:Suppressions used: 2024-03-27T08:25:41.765 INFO:teuthology.orchestra.run.smithi104.stderr: count bytes template 2024-03-27T08:25:41.765 INFO:teuthology.orchestra.run.smithi104.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:41.765 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:41.765 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:25:41.823 INFO:teuthology.orchestra.run.smithi104.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:41.827 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:41,835 seastar - Reactor backend: linux-aio 2024-03-27T08:25:41.835 INFO:teuthology.orchestra.run.smithi104.stderr:==29214==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:41.901 INFO:teuthology.orchestra.run.smithi104.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99f029 0xb871af0 0xb87306c 0xb873282 0xb8733d8 0xb8738a1 0x54daf 0x113bcd 0x4a0bd 0x35b7c 0x36eee 0x53036fe 0x5303ac9 0x5d7e5c9 0x5d819b4 0x5d81eb6 0x5bc6c6f 0x5be1d33 0x6ebc2e4 0x6ecdab7 0x6ecef54 0x6ecefd9 0x6ecf0db 0x6ecf1d3 0x6ecf3e0 0x6ecf5c6 0x6ecf915 0x6ecfcb9 0x6ecfee9 0x6ed0153 0x6ed0420 0x6ed063e 0x6ed06f6 0x6ed07bc 0x6ed089d 0x5a2d276 0x5a2d380 0x5a2d43c 0x5a2d4f8 0x5a2d5b4 0x5a2d701 0x5a2d88c 0x5a2dc4d 0x5a4a6ca 0xb884775 0xb89ea0f 0xb93fa0d 0xb94105b 0xb61d7c3 0xb61e13f 0x368058a 0x3feaf 0x3ff5f 0x346c444 2024-03-27T08:25:41.901 INFO:teuthology.orchestra.run.smithi104.stderr:kernel callstack: 2024-03-27T08:25:45.302 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:45.303 INFO:teuthology.orchestra.run.smithi104.stderr:Suppressions used: 2024-03-27T08:25:45.303 INFO:teuthology.orchestra.run.smithi104.stderr: count bytes template 2024-03-27T08:25:45.303 INFO:teuthology.orchestra.run.smithi104.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:45.303 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:45.303 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:25:45.303 INFO:teuthology.orchestra.run.smithi104.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:45.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-27T08:25:45.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:45.656 INFO:teuthology.orchestra.run.smithi104.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:45.657 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:45,666 seastar - Reactor backend: linux-aio 2024-03-27T08:25:45.662 INFO:teuthology.orchestra.run.smithi104.stderr:==30026==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:45.779 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:45,788 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:25:45.780 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:45,788 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-03-27T08:25:45.780 INFO:teuthology.orchestra.run.smithi104.stderr:DEBUG 2024-03-27 08:25:45,789 [shard 0:main] seastar - reactor::drain 2024-03-27T08:25:46.333 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:46.333 INFO:teuthology.orchestra.run.smithi104.stderr:Suppressions used: 2024-03-27T08:25:46.334 INFO:teuthology.orchestra.run.smithi104.stderr: count bytes template 2024-03-27T08:25:46.334 INFO:teuthology.orchestra.run.smithi104.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:46.334 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:46.334 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:25:46.393 INFO:teuthology.orchestra.run.smithi104.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:46.396 INFO:teuthology.orchestra.run.smithi104.stderr:INFO 2024-03-27 08:25:46,405 seastar - Reactor backend: linux-aio 2024-03-27T08:25:46.405 INFO:teuthology.orchestra.run.smithi104.stderr:==30024==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:49.860 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:49.860 INFO:teuthology.orchestra.run.smithi104.stderr:Suppressions used: 2024-03-27T08:25:49.860 INFO:teuthology.orchestra.run.smithi104.stderr: count bytes template 2024-03-27T08:25:49.860 INFO:teuthology.orchestra.run.smithi104.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:49.860 INFO:teuthology.orchestra.run.smithi104.stderr:----------------------------------------------------- 2024-03-27T08:25:49.861 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:25:49.861 INFO:teuthology.orchestra.run.smithi104.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:49.884 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-27T08:25:49.910 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-27T08:25:49.943 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-27T08:25:49.943 INFO:tasks.ceph:role: osd.2 2024-03-27T08:25:49.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:25:49.943 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-27T08:25:50.021 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:25:50.021 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:25:50.022 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:25:50.529 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-03-27T08:25:50.531 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-03-27T08:25:50.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-03-27T08:25:50.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-27T08:25:50.684 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-27T08:25:50.753 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-27T08:25:50.753 INFO:tasks.ceph:role: osd.3 2024-03-27T08:25:50.753 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:25:50.753 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:25:50.829 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:25:51.275 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-03-27T08:25:51.277 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-03-27T08:25:51.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-03-27T08:25:51.369 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-27T08:25:51.400 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:51.767 INFO:teuthology.orchestra.run.smithi177.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:51.768 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:51,777 seastar - Reactor backend: linux-aio 2024-03-27T08:25:51.782 INFO:teuthology.orchestra.run.smithi177.stderr:==28868==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:51.889 INFO:teuthology.orchestra.run.smithi177.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99f029 0xb871af0 0xb87306c 0xb873282 0xb8733d8 0xb8738a1 0x54daf 0x4a0ae 0x35b7c 0x3689e 0x5db39d7 0x5e02c0b 0x5d81750 0x5d81c84 0x5bc6c6f 0x5be1d33 0x6ebc2e4 0x6ecdab7 0x6ecef54 0x6ecefd9 0x6ecf0db 0x6ecf1d3 0x6ecf3e0 0x6ecf5c6 0x6ecf915 0x6ecfcb9 0x6ecfee9 0x6ed0153 0x6ed0420 0x6ed063e 0x6ed06f6 0x6ed07bc 0x6ed089d 0x5a2d276 0x5a2d380 0x5a2d43c 0x5a2d4f8 0x5a2d5b4 0x5a2d701 0x5a2d88c 0x5a2dc4d 0x5a4a6ca 0xb884775 0xb89ea0f 0xb93fa0d 0xb94105b 0xb61d7c3 0xb61e13f 0x37df266 0x37dfdbf 0x367fd7a 0x3feaf 0x3ff5f 0x346c444 2024-03-27T08:25:51.889 INFO:teuthology.orchestra.run.smithi177.stderr:kernel callstack: 2024-03-27T08:25:51.905 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:51,913 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:25:51.905 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:51,913 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-03-27T08:25:51.905 INFO:teuthology.orchestra.run.smithi177.stderr:DEBUG 2024-03-27 08:25:51,914 [shard 0:main] seastar - reactor::drain 2024-03-27T08:25:52.460 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:25:52.460 INFO:teuthology.orchestra.run.smithi177.stderr:Suppressions used: 2024-03-27T08:25:52.460 INFO:teuthology.orchestra.run.smithi177.stderr: count bytes template 2024-03-27T08:25:52.460 INFO:teuthology.orchestra.run.smithi177.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:52.460 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:25:52.460 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-27T08:25:52.520 INFO:teuthology.orchestra.run.smithi177.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:52.524 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:52,532 seastar - Reactor backend: linux-aio 2024-03-27T08:25:52.532 INFO:teuthology.orchestra.run.smithi177.stderr:==28864==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:52.597 INFO:teuthology.orchestra.run.smithi177.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99f029 0xb871af0 0xb87306c 0xb873282 0xb8733d8 0xb8738a1 0x54daf 0x5d9c23c 0x5e02c29 0x5d81750 0x5d8192a 0x5d81eb6 0x5bc6c6f 0x5be1d33 0x6ebc2e4 0x6ecdab7 0x6ecef54 0x6ecefd9 0x6ecf0db 0x6ecf1d3 0x6ecf3e0 0x6ecf5c6 0x6ecf915 0x6ecfcb9 0x6ecfee9 0x6ed0153 0x6ed0420 0x6ed063e 0x6ed06f6 0x6ed07bc 0x6ed089d 0x5a2d276 0x5a2d380 0x5a2d43c 0x5a2d4f8 0x5a2d5b4 0x5a2d701 0x5a2d88c 0x5a2dc4d 0x5a4a6ca 0xb884775 0xb89ea0f 0xb93fa0d 0xb94105b 0xb61d7c3 0xb61e13f 0x368058a 0x3feaf 0x3ff5f 0x346c444 2024-03-27T08:25:52.598 INFO:teuthology.orchestra.run.smithi177.stderr:kernel callstack: 2024-03-27T08:25:56.009 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:25:56.010 INFO:teuthology.orchestra.run.smithi177.stderr:Suppressions used: 2024-03-27T08:25:56.010 INFO:teuthology.orchestra.run.smithi177.stderr: count bytes template 2024-03-27T08:25:56.010 INFO:teuthology.orchestra.run.smithi177.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:56.010 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:25:56.010 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-27T08:25:56.010 INFO:teuthology.orchestra.run.smithi177.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:25:56.034 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-27T08:25:56.062 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:25:56.349 INFO:teuthology.orchestra.run.smithi177.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:56.350 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:56,359 seastar - Reactor backend: linux-aio 2024-03-27T08:25:56.355 INFO:teuthology.orchestra.run.smithi177.stderr:==29677==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:25:56.479 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:56,488 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:25:56.479 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:56,488 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-03-27T08:25:56.480 INFO:teuthology.orchestra.run.smithi177.stderr:DEBUG 2024-03-27 08:25:56,489 [shard 0:main] seastar - reactor::drain 2024-03-27T08:25:57.013 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:25:57.013 INFO:teuthology.orchestra.run.smithi177.stderr:Suppressions used: 2024-03-27T08:25:57.013 INFO:teuthology.orchestra.run.smithi177.stderr: count bytes template 2024-03-27T08:25:57.013 INFO:teuthology.orchestra.run.smithi177.stderr: 1 8 MallocExtension::Register 2024-03-27T08:25:57.013 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:25:57.013 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-27T08:25:57.073 INFO:teuthology.orchestra.run.smithi177.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:25:57.077 INFO:teuthology.orchestra.run.smithi177.stderr:INFO 2024-03-27 08:25:57,086 seastar - Reactor backend: linux-aio 2024-03-27T08:25:57.087 INFO:teuthology.orchestra.run.smithi177.stderr:==29675==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:26:00.555 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:26:00.555 INFO:teuthology.orchestra.run.smithi177.stderr:Suppressions used: 2024-03-27T08:26:00.556 INFO:teuthology.orchestra.run.smithi177.stderr: count bytes template 2024-03-27T08:26:00.556 INFO:teuthology.orchestra.run.smithi177.stderr: 1 8 MallocExtension::Register 2024-03-27T08:26:00.556 INFO:teuthology.orchestra.run.smithi177.stderr:----------------------------------------------------- 2024-03-27T08:26:00.556 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-27T08:26:00.556 INFO:teuthology.orchestra.run.smithi177.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:26:00.579 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-27T08:26:00.607 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-27T08:26:00.607 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:26:00.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-27T08:26:00.637 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:26:00.637 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-27T08:26:00.706 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:26:00.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-27T08:26:00.733 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:26:00.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-27T08:26:00.800 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:26:00.800 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-27T08:26:00.866 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:26:00.866 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-27T08:26:00.883 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:26:00.883 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-27T08:26:00.920 INFO:tasks.ceph:Adding keys to all mons... 2024-03-27T08:26:00.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.0] 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAF2ANmQ6UFOxAAMeNTwbKs5VPm237dF4By6g== 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.1] 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAK2ANm4hZBIRAA+DRFSM7gDIqta+JrRbPC7w== 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.x] 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAC2ANmbp2tNRAApg3+bgNZ6dK7ZP1+zU8ZIg== 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.2] 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAQ2ANm+SERKRAAcOZjpTbyIIEh66uiOQrN6g== 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.3] 2024-03-27T08:26:00.950 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAV2ANmxdBHDhAAaiPDaFbhuRzLB1uNJMB3qg== 2024-03-27T08:26:00.951 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2024-03-27T08:26:00.951 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAC2ANm4MzvOBAAqkkfmi5BzTeiBygp+Z7sww== 2024-03-27T08:26:00.951 INFO:teuthology.orchestra.run.smithi104.stdout:[client.1] 2024-03-27T08:26:00.951 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQAD2ANmcjVbARAAgsDl3gFolB1FZEv3+Ln3uQ== 2024-03-27T08:26:00.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:26:01.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:26:01.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-27T08:26:01.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:26:01.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:26:01.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-27T08:26:01.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-27T08:26:01.435 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-27T08:26:01.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-27T08:26:01.462 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-27T08:26:01.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.578+0000 7f3624431a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.578+0000 7f3624431a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.583 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-27T08:26:01.610 DEBUG:teuthology.orchestra.run.smithi104:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:26:01.664 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-27T08:26:01.665 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-27T08:26:01.665 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-27T08:26:01.707 INFO:tasks.ceph.mon.a:Started 2024-03-27T08:26:01.708 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-27T08:26:01.709 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-27T08:26:01.709 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-27T08:26:01.712 INFO:tasks.ceph.mgr.x:Started 2024-03-27T08:26:01.712 INFO:tasks.ceph:Setting crush tunables to default 2024-03-27T08:26:01.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd crush tunables default 2024-03-27T08:26:01.789 INFO:tasks.ceph.mon.a.smithi104.stderr:2024-03-27T08:26:01.796+0000 7fc2fc8a5a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.789 INFO:tasks.ceph.mon.a.smithi104.stderr:2024-03-27T08:26:01.797+0000 7fc2fc8a5a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.789 INFO:tasks.ceph.mon.a.smithi104.stderr:2024-03-27T08:26:01.797+0000 7fc2fc8a5a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.803 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:01.811+0000 7f6445973000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.821 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.823+0000 7fb7e006d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.830+0000 7fb7e006d640 1 Processor -- start 2024-03-27T08:26:01.823 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.830+0000 7fb7e006d640 1 -- start start 2024-03-27T08:26:01.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.831+0000 7fb7e006d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:01.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.831+0000 7fb7e006d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d809ee40 con 0x7fb7d8131730 2024-03-27T08:26:01.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.831+0000 7fb7e006d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fb7d80736e0 con 0x7fb7d81379a0 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49920/0 (socket says 172.21.15.104:49920) 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dde09640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb7d8131730 0x7fb7d8131b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33628/0 (socket says 172.21.15.104:33628) 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 -- 172.21.15.104:0/15914012 learned_addr learned my addr 172.21.15.104:0/15914012 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 -- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 msgr2=0x7fb7d8139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 -- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 msgr2=0x7fb7d8139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 --2- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-27T08:26:01.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.832+0000 7fb7dd608640 1 --2- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T08:26:01.844 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.851+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344783122 0 0) 0x7fb7d809ee40 con 0x7fb7d8131730 2024-03-27T08:26:01.844 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.851+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7c4003580 con 0x7fb7d8131730 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1053708607 0 0) 0x7fb7c4003580 con 0x7fb7d8131730 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 msgr2=0x7fb7d8139dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 --2- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d80a0850 con 0x7fb7d8131730 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 ==== 205+0+0 (unknown 4189522127 0 0) 0x7fb7d4001920 con 0x7fb7d8131730 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 >> v1:172.21.15.104:6789/0 conn(0x7fb7d8131730 legacy=0x7fb7d8131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 --2- 172.21.15.104:0/15914012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80736e0 con 0x7fb7c40039d0 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dde09640 1 --2- 172.21.15.104:0/15914012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:01.845 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.852+0000 7fb7dde09640 1 -- 172.21.15.104:0/15914012 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c4003580 con 0x7fb7c40039d0 2024-03-27T08:26:01.846 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.854+0000 7fb7dde09640 1 --2- 172.21.15.104:0/15914012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7c4003e20 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5269b56fede380 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:01.846 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.854+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb7d400a3f0 con 0x7fb7c40039d0 2024-03-27T08:26:01.846 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.854+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb7d400aa50 con 0x7fb7c40039d0 2024-03-27T08:26:01.846 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.854+0000 7fb7dce07640 1 -- 172.21.15.104:0/15914012 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb7d400acf0 con 0x7fb7c40039d0 2024-03-27T08:26:01.847 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 -- 172.21.15.104:0/15914012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 msgr2=0x7fb7c4003e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:01.847 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 --2- 172.21.15.104:0/15914012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7c4003e20 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 -- 172.21.15.104:0/15914012 shutdown_connections 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 --2- 172.21.15.104:0/15914012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7c4003e20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 --2- 172.21.15.104:0/15914012 >> v2:172.21.15.104:3300/0 conn(0x7fb7d81379a0 0x7fb7d8139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 -- 172.21.15.104:0/15914012 >> 172.21.15.104:0/15914012 conn(0x7fb7d80a4ae0 msgr2=0x7fb7d80a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 -- 172.21.15.104:0/15914012 shutdown_connections 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 1 -- 172.21.15.104:0/15914012 wait complete. 2024-03-27T08:26:01.848 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.855+0000 7fb7e006d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.849 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:01.855+0000 7f6445973000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.849 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:01.855+0000 7f6445973000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:01.849 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.856+0000 7fb7e006d640 1 Processor -- start 2024-03-27T08:26:01.849 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.856+0000 7fb7e006d640 1 -- start start 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.857+0000 7fb7e006d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7d811dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7e006d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8120570 con 0x7fb7c40039d0 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7dde09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7d811dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7dde09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7d811dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49944/0 (socket says 172.21.15.104:49944) 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7dde09640 1 -- 172.21.15.104:0/1403305366 learned_addr learned my addr 172.21.15.104:0/1403305366 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7dde09640 1 -- 172.21.15.104:0/1403305366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d811ef00 con 0x7fb7c40039d0 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7dde09640 1 --2- 172.21.15.104:0/1403305366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7d811dcc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:01.851 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb7d400aea0 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb7d4001f30 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d8121da0 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb7d400acf0 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d8120ae0 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fb7d4017430 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.858+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (crc 0 0 0) 0x7fb7d4015b00 con 0x7fb7c40039d0 2024-03-27T08:26:01.852 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.859+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d81377d0 con 0x7fb7c40039d0 2024-03-27T08:26:01.854 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.862+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fb7d81377d0 con 0x7fb7c40039d0 2024-03-27T08:26:01.917 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.925+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb7d80a28d0 con 0x7fb7c40039d0 2024-03-27T08:26:01.918 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.926+0000 7fb7ce7fc640 1 -- 172.21.15.104:0/1403305366 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (crc 0 0 0) 0x7fb7d80a28d0 con 0x7fb7c40039d0 2024-03-27T08:26:01.918 INFO:teuthology.orchestra.run.smithi104.stderr:adjusted tunables profile to default 2024-03-27T08:26:01.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 msgr2=0x7fb7d811dcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 --2- 172.21.15.104:0/1403305366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7d811dcc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 shutdown_connections 2024-03-27T08:26:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 --2- 172.21.15.104:0/1403305366 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40039d0 0x7fb7d811dcc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 >> 172.21.15.104:0/1403305366 conn(0x7fb7d80a4ae0 msgr2=0x7fb7d80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 shutdown_connections 2024-03-27T08:26:01.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:01.927+0000 7fb7e006d640 1 -- 172.21.15.104:0/1403305366 wait complete. 2024-03-27T08:26:01.930 INFO:tasks.ceph:check_enable_crimson: True 2024-03-27T08:26:01.930 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-03-27T08:26:01.930 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-03-27T08:26:02.037 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.040+0000 7f6061f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:02.038 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.046+0000 7f6061f5d640 1 Processor -- start 2024-03-27T08:26:02.038 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.046+0000 7f6061f5d640 1 -- start start 2024-03-27T08:26:02.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.047+0000 7f6061f5d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f605c1314e0 0x7f605c1318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:02.040 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.047+0000 7f6061f5d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605c06f950 con 0x7f605c131e30 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.047+0000 7f6061f5d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f605c06e6a0 con 0x7f605c1314e0 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.047+0000 7f605b7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f605c1314e0 0x7f605c1318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605affd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f605c131e30 0x7f605c12e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33636/0 (socket says 172.21.15.104:33636) 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605b7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f605c1314e0 0x7f605c1318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49956/0 (socket says 172.21.15.104:49956) 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605affd640 1 -- 172.21.15.104:0/866767327 learned_addr learned my addr 172.21.15.104:0/866767327 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605b7fe640 1 -- 172.21.15.104:0/866767327 >> v1:172.21.15.104:6789/0 conn(0x7f605c131e30 legacy=0x7f605c12e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605b7fe640 1 -- 172.21.15.104:0/866767327 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c075160 con 0x7f605c1314e0 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605a7fc640 1 -- 172.21.15.104:0/866767327 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 610010627 0 0) 0x7f605c06f950 con 0x7f605c131e30 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605b7fe640 1 --2- 172.21.15.104:0/866767327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c1318f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ae729c7113f534 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605a7fc640 1 -- 172.21.15.104:0/866767327 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f604c009d90 con 0x7f605c1314e0 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605a7fc640 1 -- 172.21.15.104:0/866767327 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f604c00e230 con 0x7f605c1314e0 2024-03-27T08:26:02.041 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.048+0000 7f605a7fc640 1 -- 172.21.15.104:0/866767327 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f604c00e4d0 con 0x7f605c1314e0 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 -- 172.21.15.104:0/866767327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 msgr2=0x7f605c1318f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 --2- 172.21.15.104:0/866767327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c1318f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 -- 172.21.15.104:0/866767327 shutdown_connections 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 --2- 172.21.15.104:0/866767327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c1318f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 -- 172.21.15.104:0/866767327 >> 172.21.15.104:0/866767327 conn(0x7f605c0a0c40 msgr2=0x7f605c0a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 -- 172.21.15.104:0/866767327 shutdown_connections 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 1 -- 172.21.15.104:0/866767327 wait complete. 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.049+0000 7f6061f5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.050+0000 7f6061f5d640 1 Processor -- start 2024-03-27T08:26:02.043 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.051+0000 7f6061f5d640 1 -- start start 2024-03-27T08:26:02.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6061f5d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c11c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:02.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6061f5d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f605c11bd00 con 0x7f605c1314e0 2024-03-27T08:26:02.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f605b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c11c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f605b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c11c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49970/0 (socket says 172.21.15.104:49970) 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f605b7fe640 1 -- 172.21.15.104:0/3977850083 learned_addr learned my addr 172.21.15.104:0/3977850083 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f605b7fe640 1 -- 172.21.15.104:0/3977850083 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c11c240 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f605b7fe640 1 --2- 172.21.15.104:0/3977850083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c11c670 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f604c013530 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f604c013690 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c11bf30 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f604c013930 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c11e7e0 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f604c0123f0 con 0x7f605c1314e0 2024-03-27T08:26:02.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.052+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (crc 0 0 0) 0x7f604c018040 con 0x7f605c1314e0 2024-03-27T08:26:02.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.053+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c063450 con 0x7f605c1314e0 2024-03-27T08:26:02.048 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.056+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f604c012880 con 0x7f605c1314e0 2024-03-27T08:26:02.121 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.128+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) -- 0x7f605c06c7d0 con 0x7f605c1314e0 2024-03-27T08:26:02.121 INFO:tasks.ceph.mon.a.smithi104.stderr:2024-03-27T08:26:02.129+0000 7fc2f35f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-03-27T08:26:02.121 INFO:tasks.ceph.mon.a.smithi104.stderr:Please be aware that this feature is experimental, untested, 2024-03-27T08:26:02.121 INFO:tasks.ceph.mon.a.smithi104.stderr:unsupported, and may result in data corruption, data loss, 2024-03-27T08:26:02.121 INFO:tasks.ceph.mon.a.smithi104.stderr:and/or irreparable damage to your cluster. Do not use 2024-03-27T08:26:02.121 INFO:tasks.ceph.mon.a.smithi104.stderr:feature with important data. 2024-03-27T08:26:02.121 INFO:tasks.ceph.mon.a.smithi104.stderr: 2024-03-27T08:26:02.330 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:02.338+0000 7f6445973000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T08:26:02.383 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:02.391+0000 7f6445973000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T08:26:02.906 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:02.914+0000 7f6445973000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T08:26:02.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.927+0000 7f6043fff640 1 -- 172.21.15.104:0/3977850083 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) ==== 99+0+0 (crc 0 0 0) 0x7f604c012a20 con 0x7f605c1314e0 2024-03-27T08:26:02.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 msgr2=0x7f605c11c670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:02.921 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 --2- 172.21.15.104:0/3977850083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c11c670 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:02.921 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 shutdown_connections 2024-03-27T08:26:02.921 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 --2- 172.21.15.104:0/3977850083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f605c1314e0 0x7f605c11c670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:02.921 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 >> 172.21.15.104:0/3977850083 conn(0x7f605c0a0c40 msgr2=0x7f605c1b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:02.921 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 shutdown_connections 2024-03-27T08:26:02.921 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:02.928+0000 7f6061f5d640 1 -- 172.21.15.104:0/3977850083 wait complete. 2024-03-27T08:26:02.933 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-27T08:26:02.934 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:26:02.934 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-27T08:26:02.959 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:02.967+0000 7f6445973000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T08:26:02.963 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:26:02.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-27T08:26:03.003 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.011+0000 7f6445973000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T08:26:03.032 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:26:03.032 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-27T08:26:03.059 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.067+0000 7f6445973000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T08:26:03.060 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T08:26:03.060 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-27T08:26:03.103 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.111+0000 7f6445973000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T08:26:03.127 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 19101a14-7c59-40ec-845d-53031c7f1fa0 0 2024-03-27T08:26:03.225 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.233+0000 7f6445973000 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T08:26:03.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.279+0000 7f459099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.277 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.285+0000 7f459099d640 1 Processor -- start 2024-03-27T08:26:03.278 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.285+0000 7f459099d640 1 -- start start 2024-03-27T08:26:03.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f459099d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f458c113670 0x7f458c113a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f459099d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c114240 con 0x7f458c1180e0 2024-03-27T08:26:03.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f459099d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f458c118c20 con 0x7f458c113670 2024-03-27T08:26:03.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f458a59c640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f458c113670 0x7f458c113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f458a59c640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f458c113670 0x7f458c113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.177:56080/0 (socket says 172.21.15.177:56080) 2024-03-27T08:26:03.279 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f458a59c640 1 -- 172.21.15.177:0/1849767665 learned_addr learned my addr 172.21.15.177:0/1849767665 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.286+0000 7f4589d9b640 1 --1- 172.21.15.177:0/1849767665 >> v1:172.21.15.104:6789/0 conn(0x7f458c1180e0 0x7f458c1184f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.177:37222/0 (socket says 172.21.15.177:37222) 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458a59c640 1 -- 172.21.15.177:0/1849767665 >> v1:172.21.15.104:6789/0 conn(0x7f458c1180e0 legacy=0x7f458c1184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458a59c640 1 -- 172.21.15.177:0/1849767665 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c118d90 con 0x7f458c113670 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458959a640 1 -- 172.21.15.177:0/1849767665 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908491611 0 0) 0x7f457c001110 con 0x7f458c1180e0 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458a59c640 1 --2- 172.21.15.177:0/1849767665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c113a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b44f3400a840fc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458959a640 1 -- 172.21.15.177:0/1849767665 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4580009e50 con 0x7f458c113670 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458959a640 1 -- 172.21.15.177:0/1849767665 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f458000e230 con 0x7f458c113670 2024-03-27T08:26:03.280 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.287+0000 7f458959a640 1 -- 172.21.15.177:0/1849767665 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f458000e4d0 con 0x7f458c113670 2024-03-27T08:26:03.281 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 -- 172.21.15.177:0/1849767665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 msgr2=0x7f458c113a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.281 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 --2- 172.21.15.177:0/1849767665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c113a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.281 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 -- 172.21.15.177:0/1849767665 shutdown_connections 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 --2- 172.21.15.177:0/1849767665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c113a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 -- 172.21.15.177:0/1849767665 >> 172.21.15.177:0/1849767665 conn(0x7f458c06dab0 msgr2=0x7f458c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 -- 172.21.15.177:0/1849767665 shutdown_connections 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 1 -- 172.21.15.177:0/1849767665 wait complete. 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.288+0000 7f459099d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.289+0000 7f459099d640 1 Processor -- start 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.289+0000 7f459099d640 1 -- start start 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.289+0000 7f459099d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c1ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.282 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.289+0000 7f459099d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1ae3e0 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.290+0000 7f458a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c1ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.290+0000 7f458a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c1ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.177:56082/0 (socket says 172.21.15.177:56082) 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.290+0000 7f458a59c640 1 -- 172.21.15.177:0/4122749367 learned_addr learned my addr 172.21.15.177:0/4122749367 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.290+0000 7f458a59c640 1 -- 172.21.15.177:0/4122749367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c1ae520 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.290+0000 7f458a59c640 1 --2- 172.21.15.177:0/4122749367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c1ade10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.290+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f458000ee60 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4580015420 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c1ae710 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f45800156c0 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1a8170 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f4580015c00 con 0x7f458c113670 2024-03-27T08:26:03.284 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 747+0+0 (crc 0 0 0) 0x7f45800116d0 con 0x7f458c113670 2024-03-27T08:26:03.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.291+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c1ae900 con 0x7f458c113670 2024-03-27T08:26:03.286 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.294+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f4580011930 con 0x7f458c113670 2024-03-27T08:26:03.300 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.308+0000 7f6445973000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T08:26:03.348 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.356+0000 7f6445973000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T08:26:03.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.366+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "19101a14-7c59-40ec-845d-53031c7f1fa0", "id": 0} v 0) -- 0x7f458c0008d0 con 0x7f458c113670 2024-03-27T08:26:03.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.368+0000 7f457affd640 1 -- 172.21.15.177:0/4122749367 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19101a14-7c59-40ec-845d-53031c7f1fa0", "id": 0}]=0 v4) ==== 112+0+2 (crc 0 0 0) 0x7f4580011ad0 con 0x7f458c113670 2024-03-27T08:26:03.360 INFO:teuthology.orchestra.run.smithi177.stdout:0 2024-03-27T08:26:03.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 msgr2=0x7f458c1ade10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 --2- 172.21.15.177:0/4122749367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c1ade10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 shutdown_connections 2024-03-27T08:26:03.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 --2- 172.21.15.177:0/4122749367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c113670 0x7f458c1ade10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.361 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 >> 172.21.15.177:0/4122749367 conn(0x7f458c06dab0 msgr2=0x7f458c117030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 shutdown_connections 2024-03-27T08:26:03.362 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.369+0000 7f459099d640 1 -- 172.21.15.177:0/4122749367 wait complete. 2024-03-27T08:26:03.372 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new e5f68525-afd8-4fe6-8c06-3eedba34bef9 1 2024-03-27T08:26:03.404 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.412+0000 7f6445973000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T08:26:03.455 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.463+0000 7f6445973000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T08:26:03.482 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.484+0000 7f266af5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.491+0000 7f266af5d640 1 Processor -- start 2024-03-27T08:26:03.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.491+0000 7f266af5d640 1 -- start start 2024-03-27T08:26:03.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.491+0000 7f266af5d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f266411c320 0x7f266411e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.483 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.491+0000 7f266af5d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266411ed10 con 0x7f2664113670 2024-03-27T08:26:03.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.491+0000 7f266af5d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f266411feb0 con 0x7f266411c320 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.491+0000 7f2663fff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f266411c320 0x7f266411e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f2668cf9640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f2664113670 0x7f2664113a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.177:37228/0 (socket says 172.21.15.177:37228) 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f2668cf9640 1 -- 172.21.15.177:0/45840553 learned_addr learned my addr 172.21.15.177:0/45840553 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f2663fff640 1 -- 172.21.15.177:0/45840553 >> v1:172.21.15.104:6789/0 conn(0x7f2664113670 legacy=0x7f2664113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f2663fff640 1 -- 172.21.15.177:0/45840553 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266411fff0 con 0x7f266411c320 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f2663fff640 1 --2- 172.21.15.177:0/45840553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f266411c320 0x7f266411e740 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcaf4e103fca972 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f26637fe640 1 -- 172.21.15.177:0/45840553 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2650009d90 con 0x7f266411c320 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f26637fe640 1 -- 172.21.15.177:0/45840553 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f265000e230 con 0x7f266411c320 2024-03-27T08:26:03.486 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.492+0000 7f26637fe640 1 -- 172.21.15.177:0/45840553 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f265000e4d0 con 0x7f266411c320 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 -- 172.21.15.177:0/45840553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f266411c320 msgr2=0x7f266411e740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 --2- 172.21.15.177:0/45840553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f266411c320 0x7f266411e740 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 -- 172.21.15.177:0/45840553 shutdown_connections 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 --2- 172.21.15.177:0/45840553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f266411c320 0x7f266411e740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 -- 172.21.15.177:0/45840553 >> 172.21.15.177:0/45840553 conn(0x7f266406dab0 msgr2=0x7f266406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 -- 172.21.15.177:0/45840553 shutdown_connections 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 1 -- 172.21.15.177:0/45840553 wait complete. 2024-03-27T08:26:03.487 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.493+0000 7f266af5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.494+0000 7f266af5d640 1 Processor -- start 2024-03-27T08:26:03.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.494+0000 7f266af5d640 1 -- start start 2024-03-27T08:26:03.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f266af5d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 0x7f26641addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f266af5d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26641ae390 con 0x7f2664113670 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f2668cf9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 0x7f26641addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f2668cf9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 0x7f26641addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.177:56100/0 (socket says 172.21.15.177:56100) 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f2668cf9640 1 -- 172.21.15.177:0/613174365 learned_addr learned my addr 172.21.15.177:0/613174365 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f2668cf9640 1 -- 172.21.15.177:0/613174365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641ae4d0 con 0x7f2664113670 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.495+0000 7f2668cf9640 1 --2- 172.21.15.177:0/613174365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 0x7f26641addc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2658005a60 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2658005bc0 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2658005e60 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641ae720 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641a81a0 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f265800cbb0 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.496+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 862+0+0 (crc 0 0 0) 0x7f265800faa0 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.497+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2664063450 con 0x7f2664113670 2024-03-27T08:26:03.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.499+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f265800fd00 con 0x7f2664113670 2024-03-27T08:26:03.564 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.571+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e5f68525-afd8-4fe6-8c06-3eedba34bef9", "id": 1} v 0) -- 0x7f266411c3f0 con 0x7f2664113670 2024-03-27T08:26:03.565 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.573+0000 7f26617fa640 1 -- 172.21.15.177:0/613174365 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5f68525-afd8-4fe6-8c06-3eedba34bef9", "id": 1}]=0 v5) ==== 112+0+2 (crc 0 0 0) 0x7f265800fea0 con 0x7f2664113670 2024-03-27T08:26:03.565 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-03-27T08:26:03.566 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 msgr2=0x7f26641addc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.566 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 --2- 172.21.15.177:0/613174365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 0x7f26641addc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.566 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 shutdown_connections 2024-03-27T08:26:03.567 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 --2- 172.21.15.177:0/613174365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2664113670 0x7f26641addc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.567 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 >> 172.21.15.177:0/613174365 conn(0x7f266406dab0 msgr2=0x7f2664111d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.567 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 shutdown_connections 2024-03-27T08:26:03.567 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.574+0000 7f266af5d640 1 -- 172.21.15.177:0/613174365 wait complete. 2024-03-27T08:26:03.577 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 9aa42594-3a8e-45e3-b71c-779890d7f880 2 2024-03-27T08:26:03.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.729+0000 7f23b6cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.735+0000 7f23b6cad640 1 Processor -- start 2024-03-27T08:26:03.727 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.735+0000 7f23b6cad640 1 -- start start 2024-03-27T08:26:03.728 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23b6cad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23b011c320 0x7f23b011e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23b6cad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23b011ed10 con 0x7f23b0113670 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23b6cad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f23b011feb0 con 0x7f23b011c320 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23affff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23b011c320 0x7f23b011e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23b4a49640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f23b0113670 0x7f23b0113a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.177:37244/0 (socket says 172.21.15.177:37244) 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23affff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f23b011c320 0x7f23b011e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.177:56104/0 (socket says 172.21.15.177:56104) 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.736+0000 7f23b4a49640 1 -- 172.21.15.177:0/4106679414 learned_addr learned my addr 172.21.15.177:0/4106679414 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23affff640 1 -- 172.21.15.177:0/4106679414 >> v1:172.21.15.104:6789/0 conn(0x7f23b0113670 legacy=0x7f23b0113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23affff640 1 -- 172.21.15.177:0/4106679414 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b011fff0 con 0x7f23b011c320 2024-03-27T08:26:03.729 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23af7fe640 1 -- 172.21.15.177:0/4106679414 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182668235 0 0) 0x7f23b011ed10 con 0x7f23b0113670 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23affff640 1 --2- 172.21.15.177:0/4106679414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b011c320 0x7f23b011e740 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239518d794d3f56 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23af7fe640 1 -- 172.21.15.177:0/4106679414 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f239c009d90 con 0x7f23b011c320 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23af7fe640 1 -- 172.21.15.177:0/4106679414 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f239c00e230 con 0x7f23b011c320 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.737+0000 7f23af7fe640 1 -- 172.21.15.177:0/4106679414 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f239c00e4d0 con 0x7f23b011c320 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 -- 172.21.15.177:0/4106679414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b011c320 msgr2=0x7f23b011e740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 --2- 172.21.15.177:0/4106679414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b011c320 0x7f23b011e740 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 -- 172.21.15.177:0/4106679414 shutdown_connections 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 --2- 172.21.15.177:0/4106679414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b011c320 0x7f23b011e740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 -- 172.21.15.177:0/4106679414 >> 172.21.15.177:0/4106679414 conn(0x7f23b006dab0 msgr2=0x7f23b006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 -- 172.21.15.177:0/4106679414 shutdown_connections 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 1 -- 172.21.15.177:0/4106679414 wait complete. 2024-03-27T08:26:03.731 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.738+0000 7f23b6cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.739+0000 7f23b6cad640 1 Processor -- start 2024-03-27T08:26:03.732 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.739+0000 7f23b6cad640 1 -- start start 2024-03-27T08:26:03.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.739+0000 7f23b6cad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 0x7f23b01adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.733 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.739+0000 7f23b6cad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01ae1e0 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b4a49640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 0x7f23b01adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b4a49640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 0x7f23b01adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.177:56106/0 (socket says 172.21.15.177:56106) 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b4a49640 1 -- 172.21.15.177:0/1739311814 learned_addr learned my addr 172.21.15.177:0/1739311814 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b4a49640 1 -- 172.21.15.177:0/1739311814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b01ae320 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b4a49640 1 --2- 172.21.15.177:0/1739311814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 0x7f23b01adc10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f23a4004b80 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f23a4004ce0 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b01ae570 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f23a4007120 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.740+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b01a7ff0 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.741+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f23a400c610 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.741+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 977+0+0 (crc 0 0 0) 0x7f23a400fc10 con 0x7f23b0113670 2024-03-27T08:26:03.734 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.741+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b01ae760 con 0x7f23b0113670 2024-03-27T08:26:03.736 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.744+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f23a400fe70 con 0x7f23b0113670 2024-03-27T08:26:03.807 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.814+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9aa42594-3a8e-45e3-b71c-779890d7f880", "id": 2} v 0) -- 0x7f23b011c3c0 con 0x7f23b0113670 2024-03-27T08:26:03.808 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.816+0000 7f23ad7fa640 1 -- 172.21.15.177:0/1739311814 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9aa42594-3a8e-45e3-b71c-779890d7f880", "id": 2}]=0 v6) ==== 112+0+2 (crc 0 0 0) 0x7f23a400f230 con 0x7f23b0113670 2024-03-27T08:26:03.808 INFO:teuthology.orchestra.run.smithi177.stdout:2 2024-03-27T08:26:03.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 msgr2=0x7f23b01adc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 --2- 172.21.15.177:0/1739311814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 0x7f23b01adc10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 shutdown_connections 2024-03-27T08:26:03.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 --2- 172.21.15.177:0/1739311814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23b0113670 0x7f23b01adc10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.809 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 >> 172.21.15.177:0/1739311814 conn(0x7f23b006dab0 msgr2=0x7f23b0111d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 shutdown_connections 2024-03-27T08:26:03.810 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.817+0000 7f23b6cad640 1 -- 172.21.15.177:0/1739311814 wait complete. 2024-03-27T08:26:03.819 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 032cfe60-0672-43ec-b43f-aba7bf450307 3 2024-03-27T08:26:03.850 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.858+0000 7f6445973000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T08:26:03.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.931+0000 7fc9abe6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.930 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.938+0000 7fc9abe6d640 1 Processor -- start 2024-03-27T08:26:03.930 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.938+0000 7fc9abe6d640 1 -- start start 2024-03-27T08:26:03.930 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.938+0000 7fc9abe6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc9a411c320 0x7fc9a411e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.930 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.938+0000 7fc9abe6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a411ed10 con 0x7fc9a4113670 2024-03-27T08:26:03.931 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.938+0000 7fc9abe6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a411feb0 con 0x7fc9a411c320 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a9408640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc9a411c320 0x7fc9a411e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a9c09640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fc9a4113670 0x7fc9a4113a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.177:37252/0 (socket says 172.21.15.177:37252) 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a9c09640 1 -- 172.21.15.177:0/344575101 learned_addr learned my addr 172.21.15.177:0/344575101 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a8c07640 1 -- 172.21.15.177:0/344575101 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347941540 0 0) 0x7fc9a411ed10 con 0x7fc9a4113670 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a8c07640 1 -- 172.21.15.177:0/344575101 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc990003580 con 0x7fc9a4113670 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a9408640 1 -- 172.21.15.177:0/344575101 >> v1:172.21.15.104:6789/0 conn(0x7fc9a4113670 legacy=0x7fc9a4113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a9408640 1 -- 172.21.15.177:0/344575101 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a411fff0 con 0x7fc9a411c320 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a9408640 1 --2- 172.21.15.177:0/344575101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a411c320 0x7fc9a411e740 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc233cc6ea4d3b4a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a8c07640 1 -- 172.21.15.177:0/344575101 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc994009d90 con 0x7fc9a411c320 2024-03-27T08:26:03.932 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a8c07640 1 -- 172.21.15.177:0/344575101 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc99400e230 con 0x7fc9a411c320 2024-03-27T08:26:03.933 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.939+0000 7fc9a8c07640 1 -- 172.21.15.177:0/344575101 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc99400e4d0 con 0x7fc9a411c320 2024-03-27T08:26:03.935 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 -- 172.21.15.177:0/344575101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a411c320 msgr2=0x7fc9a411e740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 --2- 172.21.15.177:0/344575101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a411c320 0x7fc9a411e740 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 -- 172.21.15.177:0/344575101 shutdown_connections 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 --2- 172.21.15.177:0/344575101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a411c320 0x7fc9a411e740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 -- 172.21.15.177:0/344575101 >> 172.21.15.177:0/344575101 conn(0x7fc9a406dab0 msgr2=0x7fc9a406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 -- 172.21.15.177:0/344575101 shutdown_connections 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.940+0000 7fc9abe6d640 1 -- 172.21.15.177:0/344575101 wait complete. 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.941+0000 7fc9abe6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.941+0000 7fc9abe6d640 1 Processor -- start 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.941+0000 7fc9abe6d640 1 -- start start 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9abe6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 0x7fc9a41adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9abe6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a41ae260 con 0x7fc9a4113670 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9a9c09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 0x7fc9a41adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:03.936 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9a9c09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 0x7fc9a41adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.177:56126/0 (socket says 172.21.15.177:56126) 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9a9c09640 1 -- 172.21.15.177:0/390944825 learned_addr learned my addr 172.21.15.177:0/390944825 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9a9c09640 1 -- 172.21.15.177:0/390944825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a41ae3a0 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9a9c09640 1 --2- 172.21.15.177:0/390944825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 0x7fc9a41adc90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc9a0005a60 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc9a0005bc0 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a41ae5f0 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.942+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc9a0005e60 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.943+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a41a7ff0 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.943+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fc9a000cbd0 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.943+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1092+0+0 (crc 0 0 0) 0x7fc9a000f230 con 0x7fc9a4113670 2024-03-27T08:26:03.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.943+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a41ae7e0 con 0x7fc9a4113670 2024-03-27T08:26:03.938 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:03.946+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fc9a00070b0 con 0x7fc9a4113670 2024-03-27T08:26:03.984 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:03.992+0000 7f6445973000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T08:26:04.011 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.019+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "032cfe60-0672-43ec-b43f-aba7bf450307", "id": 3} v 0) -- 0x7fc9a411c320 con 0x7fc9a4113670 2024-03-27T08:26:04.015 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.023+0000 7fc99a7fc640 1 -- 172.21.15.177:0/390944825 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "032cfe60-0672-43ec-b43f-aba7bf450307", "id": 3}]=0 v7) ==== 112+0+2 (crc 0 0 0) 0x7fc9a0007290 con 0x7fc9a4113670 2024-03-27T08:26:04.015 INFO:teuthology.orchestra.run.smithi177.stdout:3 2024-03-27T08:26:04.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 msgr2=0x7fc9a41adc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:04.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 --2- 172.21.15.177:0/390944825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 0x7fc9a41adc90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:04.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 shutdown_connections 2024-03-27T08:26:04.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 --2- 172.21.15.177:0/390944825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9a4113670 0x7fc9a41adc90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:04.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 >> 172.21.15.177:0/390944825 conn(0x7fc9a406dab0 msgr2=0x7fc9a4111d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:04.016 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 shutdown_connections 2024-03-27T08:26:04.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T08:26:04.024+0000 7fc9abe6d640 1 -- 172.21.15.177:0/390944825 wait complete. 2024-03-27T08:26:04.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-03-27T08:26:04.063 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-27T08:26:04.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-03-27T08:26:04.094 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.102+0000 7f6445973000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T08:26:04.105 INFO:tasks.ceph.osd.0:Started 2024-03-27T08:26:04.105 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-03-27T08:26:04.139 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-27T08:26:04.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-03-27T08:26:04.140 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.147+0000 7f6445973000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T08:26:04.181 INFO:tasks.ceph.osd.1:Started 2024-03-27T08:26:04.181 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-03-27T08:26:04.213 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-27T08:26:04.213 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-03-27T08:26:04.255 INFO:tasks.ceph.osd.2:Started 2024-03-27T08:26:04.255 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-03-27T08:26:04.258 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.263+0000 7f6445973000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T08:26:04.286 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-27T08:26:04.287 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-03-27T08:26:04.329 INFO:tasks.ceph.osd.3:Started 2024-03-27T08:26:04.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:26:04.358 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.366+0000 7f6445973000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T08:26:04.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.456+0000 7f51a9c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:04.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.461+0000 7f51a9c6d640 1 Processor -- start 2024-03-27T08:26:04.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.461+0000 7f51a9c6d640 1 -- start start 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a9c6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f51a4131830 0x7f51a4131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a9c6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a40735e0 con 0x7f51a4134580 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a9c6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f51a40a0260 con 0x7f51a4131830 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a37fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f51a4131830 0x7f51a4131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a2ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f51a4134580 0x7f51a4134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33640/0 (socket says 172.21.15.104:33640) 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a2ffd640 1 -- 172.21.15.104:0/878832040 learned_addr learned my addr 172.21.15.104:0/878832040 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a37fe640 1 -- 172.21.15.104:0/878832040 >> v1:172.21.15.104:6789/0 conn(0x7f51a4134580 legacy=0x7f51a4134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a37fe640 1 -- 172.21.15.104:0/878832040 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a406eb80 con 0x7f51a4131830 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a27fc640 1 -- 172.21.15.104:0/878832040 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259717531 0 0) 0x7f51a40735e0 con 0x7f51a4134580 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.462+0000 7f51a37fe640 1 --2- 172.21.15.104:0/878832040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4131c40 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57ffe8b564472b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:04.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a27fc640 1 -- 172.21.15.104:0/878832040 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5194009d90 con 0x7f51a4131830 2024-03-27T08:26:04.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a27fc640 1 -- 172.21.15.104:0/878832040 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f519400e230 con 0x7f51a4131830 2024-03-27T08:26:04.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a27fc640 1 -- 172.21.15.104:0/878832040 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f519400e4d0 con 0x7f51a4131830 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 -- 172.21.15.104:0/878832040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 msgr2=0x7f51a4131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 --2- 172.21.15.104:0/878832040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4131c40 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 -- 172.21.15.104:0/878832040 shutdown_connections 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 --2- 172.21.15.104:0/878832040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4131c40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 -- 172.21.15.104:0/878832040 >> 172.21.15.104:0/878832040 conn(0x7f51a40a4ae0 msgr2=0x7f51a40a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 -- 172.21.15.104:0/878832040 shutdown_connections 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.463+0000 7f51a9c6d640 1 -- 172.21.15.104:0/878832040 wait complete. 2024-03-27T08:26:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.464+0000 7f51a9c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:04.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.465+0000 7f51a9c6d640 1 Processor -- start 2024-03-27T08:26:04.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.465+0000 7f51a9c6d640 1 -- start start 2024-03-27T08:26:04.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.466+0000 7f51a9c6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.466+0000 7f51a9c6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4072be0 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.466+0000 7f51a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.466+0000 7f51a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49988/0 (socket says 172.21.15.104:49988) 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.466+0000 7f51a37fe640 1 -- 172.21.15.104:0/490287021 learned_addr learned my addr 172.21.15.104:0/490287021 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f51a37fe640 1 -- 172.21.15.104:0/490287021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a411e5e0 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f51a37fe640 1 --2- 172.21.15.104:0/490287021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4070c50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5194014620 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5194014780 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a4072780 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5194014a20 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a411e2d0 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f5194013400 con 0x7f51a4131830 2024-03-27T08:26:04.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.467+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7f5194019040 con 0x7f51a4131830 2024-03-27T08:26:04.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.468+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a411ef70 con 0x7f51a4131830 2024-03-27T08:26:04.462 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.470+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f5194011c10 con 0x7f51a4131830 2024-03-27T08:26:04.526 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.533+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51a4063450 con 0x7f51a4131830 2024-03-27T08:26:04.526 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.534+0000 7f518bfff640 1 -- 172.21.15.104:0/490287021 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7f5194011e60 con 0x7f51a4131830 2024-03-27T08:26:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:04.526 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":7,"fsid":"756f8b0d-eb2f-4187-9610-708a3d134344","created":"2024-03-27T08:26:01.843411+0000","modified":"2024-03-27T08:26:04.020627+0000","last_up_change":"0.000000","last_in_change":"2024-03-27T08:26:04.020627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"19101a14-7c59-40ec-845d-53031c7f1fa0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5f68525-afd8-4fe6-8c06-3eedba34bef9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9aa42594-3a8e-45e3-b71c-779890d7f880","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"032cfe60-0672-43ec-b43f-aba7bf450307","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:26:04.527 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 msgr2=0x7f51a4070c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:04.527 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 --2- 172.21.15.104:0/490287021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4070c50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:04.527 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 shutdown_connections 2024-03-27T08:26:04.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 --2- 172.21.15.104:0/490287021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51a4131830 0x7f51a4070c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:04.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 >> 172.21.15.104:0/490287021 conn(0x7f51a40a4ae0 msgr2=0x7f51a4132380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:04.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 shutdown_connections 2024-03-27T08:26:04.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:04.535+0000 7f51a9c6d640 1 -- 172.21.15.104:0/490287021 wait complete. 2024-03-27T08:26:04.533 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.542+0000 7f6445973000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T08:26:04.538 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-27T08:26:04.539 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-27T08:26:04.581 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.588+0000 7f6445973000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T08:26:04.745 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:04.753+0000 7f6445973000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T08:26:04.941 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-27T08:26:05.043 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.046+0000 7ff0596ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:05.044 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.052+0000 7ff0596ad640 1 Processor -- start 2024-03-27T08:26:05.044 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.052+0000 7ff0596ad640 1 -- start start 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.053+0000 7ff0596ad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff0540a1010 0x7ff054134790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.053+0000 7ff0596ad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff05409f2d0 con 0x7ff054134cd0 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.053+0000 7ff0596ad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7ff05406c9d0 con 0x7ff0540a1010 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff052ffd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff0540a1010 0x7ff054134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff04bfff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff054134cd0 0x7ff054131300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33650/0 (socket says 172.21.15.104:33650) 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff04bfff640 1 -- 172.21.15.104:0/3984418201 learned_addr learned my addr 172.21.15.104:0/3984418201 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff052ffd640 1 -- 172.21.15.104:0/3984418201 >> v1:172.21.15.104:6789/0 conn(0x7ff054134cd0 legacy=0x7ff054131300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff052ffd640 1 -- 172.21.15.104:0/3984418201 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05406ea80 con 0x7ff0540a1010 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff052ffd640 1 --2- 172.21.15.104:0/3984418201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff054134790 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2fdf879139502e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff0527fc640 1 -- 172.21.15.104:0/3984418201 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff044009d80 con 0x7ff0540a1010 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff0527fc640 1 -- 172.21.15.104:0/3984418201 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff04400e230 con 0x7ff0540a1010 2024-03-27T08:26:05.047 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.054+0000 7ff0527fc640 1 -- 172.21.15.104:0/3984418201 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff04400e4d0 con 0x7ff0540a1010 2024-03-27T08:26:05.048 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 -- 172.21.15.104:0/3984418201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 msgr2=0x7ff054134790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 --2- 172.21.15.104:0/3984418201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff054134790 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 -- 172.21.15.104:0/3984418201 shutdown_connections 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 --2- 172.21.15.104:0/3984418201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff054134790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 -- 172.21.15.104:0/3984418201 >> 172.21.15.104:0/3984418201 conn(0x7ff0540a15e0 msgr2=0x7ff0540a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 -- 172.21.15.104:0/3984418201 shutdown_connections 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.055+0000 7ff0596ad640 1 -- 172.21.15.104:0/3984418201 wait complete. 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.056+0000 7ff0596ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:05.049 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.057+0000 7ff0596ad640 1 Processor -- start 2024-03-27T08:26:05.050 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.057+0000 7ff0596ad640 1 -- start start 2024-03-27T08:26:05.051 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff0596ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff0541c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:05.051 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff0596ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff05411f110 con 0x7ff0540a1010 2024-03-27T08:26:05.051 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff052ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff0541c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:05.051 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff052ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff0541c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50002/0 (socket says 172.21.15.104:50002) 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff052ffd640 1 -- 172.21.15.104:0/1082199419 learned_addr learned my addr 172.21.15.104:0/1082199419 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff052ffd640 1 -- 172.21.15.104:0/1082199419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff054120440 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.058+0000 7ff052ffd640 1 --2- 172.21.15.104:0/1082199419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff0541c9900 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff044014540 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff0440146a0 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff044014940 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541208f0 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541c9560 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7ff044013790 con 0x7ff0540a1010 2024-03-27T08:26:05.052 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.059+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7ff044019040 con 0x7ff0540a1010 2024-03-27T08:26:05.053 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.060+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0541c9750 con 0x7ff0540a1010 2024-03-27T08:26:05.054 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.062+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7ff044014d00 con 0x7ff0540a1010 2024-03-27T08:26:05.116 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.123+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff054063450 con 0x7ff0540a1010 2024-03-27T08:26:05.116 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.124+0000 7ff04b7fe640 1 -- 172.21.15.104:0/1082199419 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7ff044013a40 con 0x7ff0540a1010 2024-03-27T08:26:05.116 INFO:teuthology.misc.health.smithi104.stdout: 2024-03-27T08:26:05.116 INFO:teuthology.misc.health.smithi104.stdout:{"epoch":7,"fsid":"756f8b0d-eb2f-4187-9610-708a3d134344","created":"2024-03-27T08:26:01.843411+0000","modified":"2024-03-27T08:26:04.020627+0000","last_up_change":"0.000000","last_in_change":"2024-03-27T08:26:04.020627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"19101a14-7c59-40ec-845d-53031c7f1fa0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5f68525-afd8-4fe6-8c06-3eedba34bef9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9aa42594-3a8e-45e3-b71c-779890d7f880","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"032cfe60-0672-43ec-b43f-aba7bf450307","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:26:05.117 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 msgr2=0x7ff0541c9900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:05.117 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 --2- 172.21.15.104:0/1082199419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff0541c9900 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:05.117 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 shutdown_connections 2024-03-27T08:26:05.118 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 --2- 172.21.15.104:0/1082199419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0540a1010 0x7ff0541c9900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:05.118 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 >> 172.21.15.104:0/1082199419 conn(0x7ff0540a15e0 msgr2=0x7ff0541bc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:05.118 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 shutdown_connections 2024-03-27T08:26:05.118 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:05.125+0000 7ff0596ad640 1 -- 172.21.15.104:0/1082199419 wait complete. 2024-03-27T08:26:05.127 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-03-27T08:26:05.635 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:05.643+0000 7f6445973000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T08:26:05.920 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:05.928+0000 7f6445973000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T08:26:05.975 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:05.983+0000 7f6445973000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T08:26:06.178 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:06.186+0000 7f6445973000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T08:26:07.202 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:26:07.210+0000 7f63e4adc640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-03-27T08:26:11.531 DEBUG:teuthology.orchestra.run.smithi104:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-27T08:26:11.637 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.641+0000 7facedaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:11.638 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.646+0000 7facedaad640 1 Processor -- start 2024-03-27T08:26:11.638 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.646+0000 7facedaad640 1 -- start start 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7facedaad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7face80a1010 0x7face8134790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7facedaad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face809f2d0 con 0x7face8134cd0 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7facedaad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7face806c9d0 con 0x7face80a1010 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7face77fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7face80a1010 0x7face8134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7face6ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7face8134cd0 0x7face8131300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49566/0 (socket says 172.21.15.104:49566) 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7face6ffd640 1 -- 172.21.15.104:0/2009446037 learned_addr learned my addr 172.21.15.104:0/2009446037 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7face77fe640 1 -- 172.21.15.104:0/2009446037 >> v1:172.21.15.104:6789/0 conn(0x7face8134cd0 legacy=0x7face8131300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:11.640 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.647+0000 7face77fe640 1 -- 172.21.15.104:0/2009446037 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face806ea80 con 0x7face80a1010 2024-03-27T08:26:11.641 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7face77fe640 1 --2- 172.21.15.104:0/2009446037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face8134790 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa36d0c940ebe57 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:11.641 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7face67fc640 1 -- 172.21.15.104:0/2009446037 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7facd8009d80 con 0x7face80a1010 2024-03-27T08:26:11.641 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7face67fc640 1 -- 172.21.15.104:0/2009446037 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7facd800e230 con 0x7face80a1010 2024-03-27T08:26:11.641 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7face67fc640 1 -- 172.21.15.104:0/2009446037 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7facd800e4d0 con 0x7face80a1010 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7facedaad640 1 -- 172.21.15.104:0/2009446037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 msgr2=0x7face8134790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7facedaad640 1 --2- 172.21.15.104:0/2009446037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face8134790 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7facedaad640 1 -- 172.21.15.104:0/2009446037 shutdown_connections 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.648+0000 7facedaad640 1 --2- 172.21.15.104:0/2009446037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face8134790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.649+0000 7facedaad640 1 -- 172.21.15.104:0/2009446037 >> 172.21.15.104:0/2009446037 conn(0x7face80a15e0 msgr2=0x7face80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.649+0000 7facedaad640 1 -- 172.21.15.104:0/2009446037 shutdown_connections 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.649+0000 7facedaad640 1 -- 172.21.15.104:0/2009446037 wait complete. 2024-03-27T08:26:11.642 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.649+0000 7facedaad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:11.643 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.650+0000 7facedaad640 1 Processor -- start 2024-03-27T08:26:11.643 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.650+0000 7facedaad640 1 -- start start 2024-03-27T08:26:11.644 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.651+0000 7facedaad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face811de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.651+0000 7facedaad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7face8121fe0 con 0x7face80a1010 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7face77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face811de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7face77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face811de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39346/0 (socket says 172.21.15.104:39346) 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7face77fe640 1 -- 172.21.15.104:0/4051953380 learned_addr learned my addr 172.21.15.104:0/4051953380 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7face77fe640 1 -- 172.21.15.104:0/4051953380 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face8132050 con 0x7face80a1010 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7face77fe640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face811de50 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7facd8014620 con 0x7face80a1010 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7facd8014780 con 0x7face80a1010 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81213e0 con 0x7face80a1010 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7facd8014a20 con 0x7face80a1010 2024-03-27T08:26:11.645 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.652+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81209f0 con 0x7face80a1010 2024-03-27T08:26:11.646 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.653+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 4) ==== 45686+0+0 (crc 0 0 0) 0x7facd8014bd0 con 0x7face80a1010 2024-03-27T08:26:11.646 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.653+0000 7faccffff640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7facd4038710 0x7facd403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:11.646 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.653+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2319+0+0 (crc 0 0 0) 0x7facd8019040 con 0x7face80a1010 2024-03-27T08:26:11.646 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.653+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face81321e0 con 0x7face80a1010 2024-03-27T08:26:11.646 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.654+0000 7face6ffd640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7facd4038710 0x7facd403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:11.647 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.654+0000 7face6ffd640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7facd4038710 0x7facd403ac20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:11.651 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.659+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7face81321e0 con 0x7face80a1010 2024-03-27T08:26:11.835 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.843+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7face81321e0 con 0x7face80a1010 2024-03-27T08:26:11.836 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.844+0000 7faccffff640 1 -- 172.21.15.104:0/4051953380 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+6955 (crc 0 0 0) 0x7face81321e0 con 0x7face80a1010 2024-03-27T08:26:11.836 INFO:teuthology.misc.health.smithi104.stdout: 2024-03-27T08:26:11.836 INFO:teuthology.misc.health.smithi104.stdout:{"epoch":11,"fsid":"756f8b0d-eb2f-4187-9610-708a3d134344","created":"2024-03-27T08:26:01.843411+0000","modified":"2024-03-27T08:26:11.266881+0000","last_up_change":"2024-03-27T08:26:08.257920+0000","last_in_change":"2024-03-27T08:26:04.020627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:26:10.238077+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19101a14-7c59-40ec-845d-53031c7f1fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1714017423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":1714017423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1714017423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6811","nonce":1714017423}]},"public_addr":"172.21.15.104:6804/1714017423","cluster_addr":"172.21.15.104:6805/1714017423","heartbeat_back_addr":"172.21.15.104:6810/1714017423","heartbeat_front_addr":"172.21.15.104:6811/1714017423","state":["exists","up"]},{"osd":1,"uuid":"e5f68525-afd8-4fe6-8c06-3eedba34bef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6801","nonce":1224298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1224298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6809","nonce":1224298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1224298329}]},"public_addr":"172.21.15.104:6801/1224298329","cluster_addr":"172.21.15.104:6802/1224298329","heartbeat_back_addr":"172.21.15.104:6809/1224298329","heartbeat_front_addr":"172.21.15.104:6808/1224298329","state":["exists","up"]},{"osd":2,"uuid":"9aa42594-3a8e-45e3-b71c-779890d7f880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6805","nonce":2552358937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2552358937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":2552358937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2552358937}]},"public_addr":"172.21.15.177:6805/2552358937","cluster_addr":"172.21.15.177:6804/2552358937","heartbeat_back_addr":"172.21.15.177:6813/2552358937","heartbeat_front_addr":"172.21.15.177:6814/2552358937","state":["exists","up"]},{"osd":3,"uuid":"032cfe60-0672-43ec-b43f-aba7bf450307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":929709834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":929709834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":929709834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":929709834}]},"public_addr":"172.21.15.177:6807/929709834","cluster_addr":"172.21.15.177:6806/929709834","heartbeat_back_addr":"172.21.15.177:6812/929709834","heartbeat_front_addr":"172.21.15.177:6811/929709834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:26:11.838 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7facd4038710 msgr2=0x7facd403ac20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7facd4038710 0x7facd403ac20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 msgr2=0x7face811de50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face811de50 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 shutdown_connections 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7facd4038710 0x7facd403ac20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 --2- 172.21.15.104:0/4051953380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face80a1010 0x7face811de50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 >> 172.21.15.104:0/4051953380 conn(0x7face80a15e0 msgr2=0x7face81bc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 shutdown_connections 2024-03-27T08:26:11.839 INFO:teuthology.misc.health.smithi104.stderr:2024-03-27T08:26:11.846+0000 7facedaad640 1 -- 172.21.15.104:0/4051953380 wait complete. 2024-03-27T08:26:11.848 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-03-27T08:26:11.848 INFO:tasks.ceph:Creating RBD pool 2024-03-27T08:26:11.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-27T08:26:11.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.962+0000 7f9a2de2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:11.963 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.970+0000 7f9a2de2d640 1 Processor -- start 2024-03-27T08:26:11.963 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.971+0000 7f9a2de2d640 1 -- start start 2024-03-27T08:26:11.964 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a2de2d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9a28135030 0x7f9a28139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a2de2d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a280692e0 con 0x7f9a28134510 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a2de2d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f9a2809f320 con 0x7f9a28135030 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a26ffd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f9a28135030 0x7f9a28139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a277fe640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f9a28134510 0x7f9a28134920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49578/0 (socket says 172.21.15.104:49578) 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a277fe640 1 -- 172.21.15.104:0/3663212017 learned_addr learned my addr 172.21.15.104:0/3663212017 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a26ffd640 1 -- 172.21.15.104:0/3663212017 >> v1:172.21.15.104:6789/0 conn(0x7f9a28134510 legacy=0x7f9a28134920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a26ffd640 1 -- 172.21.15.104:0/3663212017 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a28073620 con 0x7f9a28135030 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a26ffd640 1 --2- 172.21.15.104:0/3663212017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28135030 0x7f9a28139a40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6dba607e17f244 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a267fc640 1 -- 172.21.15.104:0/3663212017 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a18009d90 con 0x7f9a28135030 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a267fc640 1 -- 172.21.15.104:0/3663212017 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9a1800e230 con 0x7f9a28135030 2024-03-27T08:26:11.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.972+0000 7f9a267fc640 1 -- 172.21.15.104:0/3663212017 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a1800e4d0 con 0x7f9a28135030 2024-03-27T08:26:11.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 -- 172.21.15.104:0/3663212017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28135030 msgr2=0x7f9a28139a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:11.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 --2- 172.21.15.104:0/3663212017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28135030 0x7f9a28139a40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 -- 172.21.15.104:0/3663212017 shutdown_connections 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 --2- 172.21.15.104:0/3663212017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28135030 0x7f9a28139a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 -- 172.21.15.104:0/3663212017 >> 172.21.15.104:0/3663212017 conn(0x7f9a280a15e0 msgr2=0x7f9a280a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 -- 172.21.15.104:0/3663212017 shutdown_connections 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 1 -- 172.21.15.104:0/3663212017 wait complete. 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.973+0000 7f9a2de2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.975+0000 7f9a2de2d640 1 Processor -- start 2024-03-27T08:26:11.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.975+0000 7f9a2de2d640 1 -- start start 2024-03-27T08:26:11.969 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.976+0000 7f9a2de2d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 0x7f9a2811d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:11.969 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.976+0000 7f9a2de2d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28120c70 con 0x7f9a28134510 2024-03-27T08:26:11.969 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.976+0000 7f9a277fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 0x7f9a2811d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.976+0000 7f9a277fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 0x7f9a2811d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39368/0 (socket says 172.21.15.104:39368) 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.976+0000 7f9a277fe640 1 -- 172.21.15.104:0/2329515710 learned_addr learned my addr 172.21.15.104:0/2329515710 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a277fe640 1 -- 172.21.15.104:0/2329515710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2811f700 con 0x7f9a28134510 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a277fe640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 0x7f9a2811d1c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a1c005a60 con 0x7f9a28134510 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9a1c005bc0 con 0x7f9a28134510 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a28120e80 con 0x7f9a28134510 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a1c005e60 con 0x7f9a28134510 2024-03-27T08:26:11.970 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.977+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a28130db0 con 0x7f9a28134510 2024-03-27T08:26:11.971 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.978+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 4) ==== 45686+0+0 (crc 0 0 0) 0x7f9a1c00c780 con 0x7f9a28134510 2024-03-27T08:26:11.971 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.978+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a28063450 con 0x7f9a28134510 2024-03-27T08:26:11.971 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.978+0000 7f9a0ffff640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9a14038710 0x7f9a1403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:11.971 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.978+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2319+0+0 (crc 0 0 0) 0x7f9a1c047c30 con 0x7f9a28134510 2024-03-27T08:26:11.972 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.978+0000 7f9a26ffd640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9a14038710 0x7f9a1403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:11.972 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.979+0000 7f9a26ffd640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9a14038710 0x7f9a1403ac20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:11.976 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:11.984+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f9a1c00d330 con 0x7f9a28134510 2024-03-27T08:26:12.159 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.166+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f9a2806c980 con 0x7f9a28134510 2024-03-27T08:26:12.266 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.273+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f9a1c0070b0 con 0x7f9a28134510 2024-03-27T08:26:12.268 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.276+0000 7f9a0ffff640 1 -- 172.21.15.104:0/2329515710 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) ==== 109+0+0 (crc 0 0 0) 0x7f9a1c00ada0 con 0x7f9a28134510 2024-03-27T08:26:12.268 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'rbd' created 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9a14038710 msgr2=0x7f9a1403ac20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9a14038710 0x7f9a1403ac20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 msgr2=0x7f9a2811d1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 0x7f9a2811d1c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 shutdown_connections 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9a14038710 0x7f9a1403ac20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 --2- 172.21.15.104:0/2329515710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a28134510 0x7f9a2811d1c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.278+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 >> 172.21.15.104:0/2329515710 conn(0x7f9a280a15e0 msgr2=0x7f9a281bc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.279+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 shutdown_connections 2024-03-27T08:26:12.272 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.279+0000 7f9a2de2d640 1 -- 172.21.15.104:0/2329515710 wait complete. 2024-03-27T08:26:12.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-27T08:26:12.418 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.416+0000 7fdc98dc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:12.419 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.427+0000 7fdc98dc6640 1 Processor -- start 2024-03-27T08:26:12.419 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.427+0000 7fdc98dc6640 1 -- start start 2024-03-27T08:26:12.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc98dc6640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc940a2ce0 0x7fdc94134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc98dc6640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc94134770 con 0x7fdc94135100 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc98dc6640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fdc940a26e0 con 0x7fdc940a2ce0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc9259c640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc940a2ce0 0x7fdc94134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc89d9b640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fdc94135100 0x7fdc940a0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49590/0 (socket says 172.21.15.104:49590) 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc9259c640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdc940a2ce0 0x7fdc94134bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39376/0 (socket says 172.21.15.104:39376) 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc89d9b640 1 -- 172.21.15.104:0/153562671 learned_addr learned my addr 172.21.15.104:0/153562671 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc9259c640 1 -- 172.21.15.104:0/153562671 >> v1:172.21.15.104:6789/0 conn(0x7fdc94135100 legacy=0x7fdc940a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc9259c640 1 -- 172.21.15.104:0/153562671 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc940a2220 con 0x7fdc940a2ce0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc9259c640 1 --2- 172.21.15.104:0/153562671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc94134bc0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28830e294621bfb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.428+0000 7fdc91d9b640 1 -- 172.21.15.104:0/153562671 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc84009d90 con 0x7fdc940a2ce0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc91d9b640 1 -- 172.21.15.104:0/153562671 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdc8400e230 con 0x7fdc940a2ce0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc91d9b640 1 -- 172.21.15.104:0/153562671 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc8400e4d0 con 0x7fdc940a2ce0 2024-03-27T08:26:12.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 -- 172.21.15.104:0/153562671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 msgr2=0x7fdc94134bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 --2- 172.21.15.104:0/153562671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc94134bc0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 -- 172.21.15.104:0/153562671 shutdown_connections 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 --2- 172.21.15.104:0/153562671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc94134bc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 -- 172.21.15.104:0/153562671 >> 172.21.15.104:0/153562671 conn(0x7fdc940a4970 msgr2=0x7fdc940a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 -- 172.21.15.104:0/153562671 shutdown_connections 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.429+0000 7fdc98dc6640 1 -- 172.21.15.104:0/153562671 wait complete. 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.430+0000 7fdc98dc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.431+0000 7fdc98dc6640 1 Processor -- start 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.431+0000 7fdc98dc6640 1 -- start start 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.432+0000 7fdc98dc6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc941c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:12.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.432+0000 7fdc98dc6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941203a0 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.432+0000 7fdc9259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc941c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.432+0000 7fdc9259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc941c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39392/0 (socket says 172.21.15.104:39392) 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.432+0000 7fdc9259c640 1 -- 172.21.15.104:0/2119296305 learned_addr learned my addr 172.21.15.104:0/2119296305 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc9259c640 1 -- 172.21.15.104:0/2119296305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc94120910 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc9259c640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc941c9a80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc84014620 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdc84014780 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc94120b40 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc84014a20 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.433+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc94121e50 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.434+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fdc84014bd0 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.434+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc94121cb0 con 0x7fdc940a2ce0 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.434+0000 7fdc8b7fe640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdc740387a0 0x7fdc7403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:12.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.435+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2661+0+0 (crc 0 0 0) 0x7fdc84019040 con 0x7fdc940a2ce0 2024-03-27T08:26:12.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.435+0000 7fdc89d9b640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdc740387a0 0x7fdc7403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:12.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.435+0000 7fdc89d9b640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdc740387a0 0x7fdc7403acb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:12.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.440+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fdc94121cb0 con 0x7fdc940a2ce0 2024-03-27T08:26:12.615 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:12.622+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7fdc94121cb0 con 0x7fdc940a2ce0 2024-03-27T08:26:13.275 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.282+0000 7fdc8b7fe640 1 -- 172.21.15.104:0/2119296305 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v13) ==== 173+0+0 (crc 0 0 0) 0x7fdc94121cb0 con 0x7fdc940a2ce0 2024-03-27T08:26:13.275 INFO:teuthology.orchestra.run.smithi104.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-27T08:26:13.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdc740387a0 msgr2=0x7fdc7403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdc740387a0 0x7fdc7403acb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 msgr2=0x7fdc941c9a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc941c9a80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 shutdown_connections 2024-03-27T08:26:13.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdc740387a0 0x7fdc7403acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.285+0000 7fdc98dc6640 1 --2- 172.21.15.104:0/2119296305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc940a2ce0 0x7fdc941c9a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.286+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 >> 172.21.15.104:0/2119296305 conn(0x7fdc940a4970 msgr2=0x7fdc941bc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:13.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.286+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 shutdown_connections 2024-03-27T08:26:13.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.286+0000 7fdc98dc6640 1 -- 172.21.15.104:0/2119296305 wait complete. 2024-03-27T08:26:13.298 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-27T08:26:13.298 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-27T08:26:13.298 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-27T08:26:13.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-27T08:26:13.299 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-27T08:26:13.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.429+0000 7fe67902d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:13.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.434+0000 7fe67902d640 1 Processor -- start 2024-03-27T08:26:13.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.434+0000 7fe67902d640 1 -- start start 2024-03-27T08:26:13.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe67902d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6741314e0 0x7fe6741318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:13.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe67902d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe67406f950 con 0x7fe674131e30 2024-03-27T08:26:13.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe67902d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fe67406e6a0 con 0x7fe6741314e0 2024-03-27T08:26:13.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe672d9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6741314e0 0x7fe6741318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe672d9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fe6741314e0 0x7fe6741318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39406/0 (socket says 172.21.15.104:39406) 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe672d9d640 1 -- 172.21.15.104:0/2507556900 learned_addr learned my addr 172.21.15.104:0/2507556900 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe66bfff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fe674131e30 0x7fe67412e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49600/0 (socket says 172.21.15.104:49600) 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe672d9d640 1 -- 172.21.15.104:0/2507556900 >> v1:172.21.15.104:6789/0 conn(0x7fe674131e30 legacy=0x7fe67412e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe672d9d640 1 -- 172.21.15.104:0/2507556900 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe674075160 con 0x7fe6741314e0 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe67259c640 1 -- 172.21.15.104:0/2507556900 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401056305 0 0) 0x7fe67406f950 con 0x7fe674131e30 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe672d9d640 1 --2- 172.21.15.104:0/2507556900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe6741318f0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716445c016af260e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.435+0000 7fe67259c640 1 -- 172.21.15.104:0/2507556900 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe664009e50 con 0x7fe6741314e0 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67259c640 1 -- 172.21.15.104:0/2507556900 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe66400e230 con 0x7fe6741314e0 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67259c640 1 -- 172.21.15.104:0/2507556900 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe66400e4d0 con 0x7fe6741314e0 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 -- 172.21.15.104:0/2507556900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 msgr2=0x7fe6741318f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 --2- 172.21.15.104:0/2507556900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe6741318f0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 -- 172.21.15.104:0/2507556900 shutdown_connections 2024-03-27T08:26:13.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 --2- 172.21.15.104:0/2507556900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe6741318f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 -- 172.21.15.104:0/2507556900 >> 172.21.15.104:0/2507556900 conn(0x7fe6740a0c40 msgr2=0x7fe6740a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:13.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 -- 172.21.15.104:0/2507556900 shutdown_connections 2024-03-27T08:26:13.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.436+0000 7fe67902d640 1 -- 172.21.15.104:0/2507556900 wait complete. 2024-03-27T08:26:13.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.437+0000 7fe67902d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.438+0000 7fe67902d640 1 Processor -- start 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.438+0000 7fe67902d640 1 -- start start 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe67902d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe67411aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe67902d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe67411d6a0 con 0x7fe6741314e0 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe672d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe67411aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe672d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe67411aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39412/0 (socket says 172.21.15.104:39412) 2024-03-27T08:26:13.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe672d9d640 1 -- 172.21.15.104:0/3797619120 learned_addr learned my addr 172.21.15.104:0/3797619120 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe672d9d640 1 -- 172.21.15.104:0/3797619120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67411c1c0 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.439+0000 7fe672d9d640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe67411aea0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.440+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe66400edc0 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.440+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe664015420 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.440+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67411d1d0 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.440+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe6640156c0 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.440+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67411d9d0 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.441+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe674063450 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.441+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe664015870 con 0x7fe6741314e0 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.441+0000 7fe66b7fe640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe6540387a0 0x7fe65403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:13.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.441+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7fe664049600 con 0x7fe6741314e0 2024-03-27T08:26:13.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.441+0000 7fe66bfff640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe6540387a0 0x7fe65403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:13.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.442+0000 7fe66bfff640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe6540387a0 0x7fe65403acb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:13.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.446+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fe664010c00 con 0x7fe6741314e0 2024-03-27T08:26:13.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.670+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe67406c7d0 con 0x7fe6741314e0 2024-03-27T08:26:13.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.672+0000 7fe66b7fe640 1 -- 172.21.15.104:0/3797619120 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+87600 (crc 0 0 0) 0x7fe664010820 con 0x7fe6741314e0 2024-03-27T08:26:13.666 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe6540387a0 msgr2=0x7fe65403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe6540387a0 0x7fe65403acb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 msgr2=0x7fe67411aea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe67411aea0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 shutdown_connections 2024-03-27T08:26:13.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe6540387a0 0x7fe65403acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 --2- 172.21.15.104:0/3797619120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6741314e0 0x7fe67411aea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 >> 172.21.15.104:0/3797619120 conn(0x7fe6740a0c40 msgr2=0x7fe67412fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:13.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 shutdown_connections 2024-03-27T08:26:13.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.674+0000 7fe67902d640 1 -- 172.21.15.104:0/3797619120 wait complete. 2024-03-27T08:26:13.677 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":769853243},{"type":"v1","addr":"172.21.15.177:6801","nonce":769853243}]},"active_addr":"172.21.15.177:6801/769853243","active_change":"2024-03-27T08:26:06.200144+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2626147879}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":518409521}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":725369326}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3094714276}]}]} 2024-03-27T08:26:13.680 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-27T08:26:13.680 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-27T08:26:13.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:26:13.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.793+0000 7fb1ee2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:13.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.798+0000 7fb1ee2ad640 1 Processor -- start 2024-03-27T08:26:13.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.798+0000 7fb1ee2ad640 1 -- start start 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1ee2ad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb1e81378c0 0x7fb1e8139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1ee2ad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e80a06a0 con 0x7fb1e8131650 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1ee2ad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e8132170 con 0x7fb1e81378c0 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e7fff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb1e8131650 0x7fb1e8131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49608/0 (socket says 172.21.15.104:49608) 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e7fff640 1 -- 172.21.15.104:0/1827018487 learned_addr learned my addr 172.21.15.104:0/1827018487 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e77fe640 1 --2- 172.21.15.104:0/1827018487 >> v2:172.21.15.104:3300/0 conn(0x7fb1e81378c0 0x7fb1e8139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e77fe640 1 -- 172.21.15.104:0/1827018487 >> v1:172.21.15.104:6789/0 conn(0x7fb1e8131650 legacy=0x7fb1e8131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e77fe640 1 -- 172.21.15.104:0/1827018487 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e806de30 con 0x7fb1e81378c0 2024-03-27T08:26:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e77fe640 1 --2- 172.21.15.104:0/1827018487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e81378c0 0x7fb1e8139ce0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256fecac98775910 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e6ffd640 1 -- 172.21.15.104:0/1827018487 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb1dc009d90 con 0x7fb1e81378c0 2024-03-27T08:26:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e6ffd640 1 -- 172.21.15.104:0/1827018487 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb1dc00e230 con 0x7fb1e81378c0 2024-03-27T08:26:13.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.799+0000 7fb1e6ffd640 1 -- 172.21.15.104:0/1827018487 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb1dc00e4d0 con 0x7fb1e81378c0 2024-03-27T08:26:13.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/1827018487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e81378c0 msgr2=0x7fb1e8139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 --2- 172.21.15.104:0/1827018487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e81378c0 0x7fb1e8139ce0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/1827018487 shutdown_connections 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 --2- 172.21.15.104:0/1827018487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e81378c0 0x7fb1e8139ce0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/1827018487 >> 172.21.15.104:0/1827018487 conn(0x7fb1e80a15e0 msgr2=0x7fb1e80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/1827018487 shutdown_connections 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/1827018487 wait complete. 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.800+0000 7fb1ee2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.802+0000 7fb1ee2ad640 1 Processor -- start 2024-03-27T08:26:13.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.802+0000 7fb1ee2ad640 1 -- start start 2024-03-27T08:26:13.796 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.803+0000 7fb1ee2ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 0x7fb1e81c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:13.796 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.803+0000 7fb1ee2ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8121cc0 con 0x7fb1e8131650 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.803+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 0x7fb1e81c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.803+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 0x7fb1e81c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39424/0 (socket says 172.21.15.104:39424) 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.803+0000 7fb1e7fff640 1 -- 172.21.15.104:0/4182101184 learned_addr learned my addr 172.21.15.104:0/4182101184 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.803+0000 7fb1e7fff640 1 -- 172.21.15.104:0/4182101184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e8120950 con 0x7fb1e8131650 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.804+0000 7fb1e7fff640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 0x7fb1e81c6290 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.804+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb1d8005a60 con 0x7fb1e8131650 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.804+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb1d8005bc0 con 0x7fb1e8131650 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.804+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e81c5be0 con 0x7fb1e8131650 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.804+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb1d8005e60 con 0x7fb1e8131650 2024-03-27T08:26:13.797 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.804+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e81200f0 con 0x7fb1e8131650 2024-03-27T08:26:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.805+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e81c6090 con 0x7fb1e8131650 2024-03-27T08:26:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.805+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fb1d800c780 con 0x7fb1e8131650 2024-03-27T08:26:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.805+0000 7fb1e4ff9640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb1d40387a0 0x7fb1d403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.805+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7fb1d8047fb0 con 0x7fb1e8131650 2024-03-27T08:26:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.806+0000 7fb1e77fe640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb1d40387a0 0x7fb1d403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.806+0000 7fb1e77fe640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb1d40387a0 0x7fb1d403acb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:13.802 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.810+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fb1d800d330 con 0x7fb1e8131650 2024-03-27T08:26:13.977 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.985+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1e8063450 con 0x7fb1e8131650 2024-03-27T08:26:13.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.986+0000 7fb1e4ff9640 1 -- 172.21.15.104:0/4182101184 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+8747 (crc 0 0 0) 0x7fb1d800d150 con 0x7fb1e8131650 2024-03-27T08:26:13.978 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:13.979 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":13,"fsid":"756f8b0d-eb2f-4187-9610-708a3d134344","created":"2024-03-27T08:26:01.843411+0000","modified":"2024-03-27T08:26:13.277434+0000","last_up_change":"2024-03-27T08:26:08.257920+0000","last_in_change":"2024-03-27T08:26:04.020627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:26:10.238077+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-27T08:26:12.210520+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19101a14-7c59-40ec-845d-53031c7f1fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1714017423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":1714017423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1714017423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6811","nonce":1714017423}]},"public_addr":"172.21.15.104:6804/1714017423","cluster_addr":"172.21.15.104:6805/1714017423","heartbeat_back_addr":"172.21.15.104:6810/1714017423","heartbeat_front_addr":"172.21.15.104:6811/1714017423","state":["exists","up"]},{"osd":1,"uuid":"e5f68525-afd8-4fe6-8c06-3eedba34bef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6801","nonce":1224298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1224298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6809","nonce":1224298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1224298329}]},"public_addr":"172.21.15.104:6801/1224298329","cluster_addr":"172.21.15.104:6802/1224298329","heartbeat_back_addr":"172.21.15.104:6809/1224298329","heartbeat_front_addr":"172.21.15.104:6808/1224298329","state":["exists","up"]},{"osd":2,"uuid":"9aa42594-3a8e-45e3-b71c-779890d7f880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6805","nonce":2552358937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2552358937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":2552358937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2552358937}]},"public_addr":"172.21.15.177:6805/2552358937","cluster_addr":"172.21.15.177:6804/2552358937","heartbeat_back_addr":"172.21.15.177:6813/2552358937","heartbeat_front_addr":"172.21.15.177:6814/2552358937","state":["exists","up"]},{"osd":3,"uuid":"032cfe60-0672-43ec-b43f-aba7bf450307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":929709834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":929709834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":929709834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":929709834}]},"public_addr":"172.21.15.177:6807/929709834","cluster_addr":"172.21.15.177:6806/929709834","heartbeat_back_addr":"172.21.15.177:6812/929709834","heartbeat_front_addr":"172.21.15.177:6811/929709834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb1d40387a0 msgr2=0x7fb1d403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb1d40387a0 0x7fb1d403acb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 msgr2=0x7fb1e81c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 0x7fb1e81c6290 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 shutdown_connections 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb1d40387a0 0x7fb1d403acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 --2- 172.21.15.104:0/4182101184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e8131650 0x7fb1e81c6290 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 >> 172.21.15.104:0/4182101184 conn(0x7fb1e80a15e0 msgr2=0x7fb1e80a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 shutdown_connections 2024-03-27T08:26:13.981 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:13.988+0000 7fb1ee2ad640 1 -- 172.21.15.104:0/4182101184 wait complete. 2024-03-27T08:26:13.991 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-27T08:26:13.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:26:14.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.103+0000 7f81d09f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.107+0000 7f81d09f5640 1 Processor -- start 2024-03-27T08:26:14.100 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.107+0000 7f81d09f5640 1 -- start start 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81d09f5640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f81cc1378c0 0x7f81cc139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81d09f5640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc0a06a0 con 0x7f81cc131650 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81d09f5640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc132170 con 0x7f81cc1378c0 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c9d9b640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f81cc1378c0 0x7f81cc139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c9d9b640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f81cc1378c0 0x7f81cc139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39426/0 (socket says 172.21.15.104:39426) 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81ca59c640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f81cc131650 0x7f81cc131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49618/0 (socket says 172.21.15.104:49618) 2024-03-27T08:26:14.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c9d9b640 1 -- 172.21.15.104:0/3018047299 learned_addr learned my addr 172.21.15.104:0/3018047299 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c9d9b640 1 -- 172.21.15.104:0/3018047299 >> v1:172.21.15.104:6789/0 conn(0x7f81cc131650 legacy=0x7f81cc131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c9d9b640 1 -- 172.21.15.104:0/3018047299 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc06de30 con 0x7f81cc1378c0 2024-03-27T08:26:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c9d9b640 1 --2- 172.21.15.104:0/3018047299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc1378c0 0x7f81cc139ce0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f6291be833f9d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c959a640 1 -- 172.21.15.104:0/3018047299 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f81c0009e50 con 0x7f81cc1378c0 2024-03-27T08:26:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c959a640 1 -- 172.21.15.104:0/3018047299 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f81c000e230 con 0x7f81cc1378c0 2024-03-27T08:26:14.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.109+0000 7f81c959a640 1 -- 172.21.15.104:0/3018047299 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f81c000e4d0 con 0x7f81cc1378c0 2024-03-27T08:26:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 -- 172.21.15.104:0/3018047299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc1378c0 msgr2=0x7f81cc139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 --2- 172.21.15.104:0/3018047299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc1378c0 0x7f81cc139ce0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 -- 172.21.15.104:0/3018047299 shutdown_connections 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 --2- 172.21.15.104:0/3018047299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc1378c0 0x7f81cc139ce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 -- 172.21.15.104:0/3018047299 >> 172.21.15.104:0/3018047299 conn(0x7f81cc0a15e0 msgr2=0x7f81cc0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 -- 172.21.15.104:0/3018047299 shutdown_connections 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.110+0000 7f81d09f5640 1 -- 172.21.15.104:0/3018047299 wait complete. 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.111+0000 7f81d09f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.112+0000 7f81d09f5640 1 Processor -- start 2024-03-27T08:26:14.105 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.112+0000 7f81d09f5640 1 -- start start 2024-03-27T08:26:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81d09f5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 0x7f81cc1c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81d09f5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc121cc0 con 0x7f81cc131650 2024-03-27T08:26:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 0x7f81cc1c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 0x7f81cc1c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39442/0 (socket says 172.21.15.104:39442) 2024-03-27T08:26:14.107 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81ca59c640 1 -- 172.21.15.104:0/1012554711 learned_addr learned my addr 172.21.15.104:0/1012554711 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81ca59c640 1 -- 172.21.15.104:0/1012554711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc120950 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.113+0000 7f81ca59c640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 0x7f81cc1c6290 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.114+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f81bc0071e0 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.114+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f81bc007340 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.114+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc1c5be0 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.114+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f81bc0075e0 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.114+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc1200f0 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.114+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f81bc007820 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.115+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc1c6090 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.115+0000 7f81baffd640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f81ac038750 0x7f81ac03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.115+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7f81bc048b50 con 0x7f81cc131650 2024-03-27T08:26:14.108 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.115+0000 7f81c9d9b640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f81ac038750 0x7f81ac03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.109 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.116+0000 7f81c9d9b640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f81ac038750 0x7f81ac03ac60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.112 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.120+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f81bc009470 con 0x7f81cc131650 2024-03-27T08:26:14.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.295+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81cc063450 con 0x7f81cc131650 2024-03-27T08:26:14.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.297+0000 7f81baffd640 1 -- 172.21.15.104:0/1012554711 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+8747 (crc 0 0 0) 0x7f81bc0090b0 con 0x7f81cc131650 2024-03-27T08:26:14.290 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:14.290 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":13,"fsid":"756f8b0d-eb2f-4187-9610-708a3d134344","created":"2024-03-27T08:26:01.843411+0000","modified":"2024-03-27T08:26:13.277434+0000","last_up_change":"2024-03-27T08:26:08.257920+0000","last_in_change":"2024-03-27T08:26:04.020627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:26:10.238077+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-27T08:26:12.210520+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19101a14-7c59-40ec-845d-53031c7f1fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1714017423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":1714017423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1714017423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6811","nonce":1714017423}]},"public_addr":"172.21.15.104:6804/1714017423","cluster_addr":"172.21.15.104:6805/1714017423","heartbeat_back_addr":"172.21.15.104:6810/1714017423","heartbeat_front_addr":"172.21.15.104:6811/1714017423","state":["exists","up"]},{"osd":1,"uuid":"e5f68525-afd8-4fe6-8c06-3eedba34bef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6801","nonce":1224298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1224298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6809","nonce":1224298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1224298329}]},"public_addr":"172.21.15.104:6801/1224298329","cluster_addr":"172.21.15.104:6802/1224298329","heartbeat_back_addr":"172.21.15.104:6809/1224298329","heartbeat_front_addr":"172.21.15.104:6808/1224298329","state":["exists","up"]},{"osd":2,"uuid":"9aa42594-3a8e-45e3-b71c-779890d7f880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6805","nonce":2552358937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2552358937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":2552358937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2552358937}]},"public_addr":"172.21.15.177:6805/2552358937","cluster_addr":"172.21.15.177:6804/2552358937","heartbeat_back_addr":"172.21.15.177:6813/2552358937","heartbeat_front_addr":"172.21.15.177:6814/2552358937","state":["exists","up"]},{"osd":3,"uuid":"032cfe60-0672-43ec-b43f-aba7bf450307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":929709834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":929709834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":929709834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":929709834}]},"public_addr":"172.21.15.177:6807/929709834","cluster_addr":"172.21.15.177:6806/929709834","heartbeat_back_addr":"172.21.15.177:6812/929709834","heartbeat_front_addr":"172.21.15.177:6811/929709834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:26:14.292 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.299+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f81ac038750 msgr2=0x7f81ac03ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.299+0000 7f81d09f5640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f81ac038750 0x7f81ac03ac60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.299+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 msgr2=0x7f81cc1c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 0x7f81cc1c6290 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 shutdown_connections 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f81ac038750 0x7f81ac03ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 --2- 172.21.15.104:0/1012554711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81cc131650 0x7f81cc1c6290 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 >> 172.21.15.104:0/1012554711 conn(0x7f81cc0a15e0 msgr2=0x7f81cc0a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 shutdown_connections 2024-03-27T08:26:14.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.300+0000 7f81d09f5640 1 -- 172.21.15.104:0/1012554711 wait complete. 2024-03-27T08:26:14.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-27T08:26:14.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-27T08:26:14.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-27T08:26:14.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-27T08:26:14.423 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.426+0000 7f8606e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.426+0000 7f7b1186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.426+0000 7ff893d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.426+0000 7fb9412ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7f7b1186d640 1 Processor -- start 2024-03-27T08:26:14.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7ff893d9d640 1 Processor -- start 2024-03-27T08:26:14.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7f7b1186d640 1 -- start start 2024-03-27T08:26:14.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7f8606e2d640 1 Processor -- start 2024-03-27T08:26:14.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7f8606e2d640 1 -- start start 2024-03-27T08:26:14.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7fb9412ad640 1 Processor -- start 2024-03-27T08:26:14.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7fb9412ad640 1 -- start start 2024-03-27T08:26:14.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.432+0000 7ff893d9d640 1 -- start start 2024-03-27T08:26:14.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f7b1186d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7b0c135860 0x7f7b0c137c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f7b0affd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7b0c071200 0x7f7b0c071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49624/0 (socket says 172.21.15.104:49624) 2024-03-27T08:26:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f7b0affd640 1 -- 172.21.15.104:0/70475475 learned_addr learned my addr 172.21.15.104:0/70475475 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0c0735d0 con 0x7f7b0c071200 2024-03-27T08:26:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0c072920 con 0x7f7b0c135860 2024-03-27T08:26:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f7b0a7fc640 1 --2- 172.21.15.104:0/70475475 >> v2:172.21.15.104:3300/0 conn(0x7f7b0c135860 0x7f7b0c137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7fb9412ad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb93c12f5e0 0x7fb93c12f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f8604bc9640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f8600131650 0x7f8600131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49638/0 (socket says 172.21.15.104:49638) 2024-03-27T08:26:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.434+0000 7f8604bc9640 1 -- 172.21.15.104:0/2578100967 learned_addr learned my addr 172.21.15.104:0/2578100967 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb93affd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb93c12f5e0 0x7fb93c12f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272675790 0 0) 0x7f7b0c0735d0 con 0x7f7b0c071200 2024-03-27T08:26:14.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af4003580 con 0x7f7b0c071200 2024-03-27T08:26:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb93affd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb93c12f5e0 0x7fb93c12f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39456/0 (socket says 172.21.15.104:39456) 2024-03-27T08:26:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb93affd640 1 -- 172.21.15.104:0/2745587395 learned_addr learned my addr 172.21.15.104:0/2745587395 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c073520 con 0x7fb93c0706b0 2024-03-27T08:26:14.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff893d9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff88c1358c0 0x7ff88c137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fb93c071880 con 0x7fb93c12f5e0 2024-03-27T08:26:14.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff893d9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c073520 con 0x7ff88c12f5e0 2024-03-27T08:26:14.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f8606e2d640 1 --2- 172.21.15.104:0/2578100967 >> v2:172.21.15.104:3300/0 conn(0x7f86001378c0 0x7f8600139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff893d9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7ff88c071880 con 0x7ff88c1358c0 2024-03-27T08:26:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff891338640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff88c1358c0 0x7ff88c137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 <== mon.0 v1:172.21.15.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3681079943 0 0) 0x7f7af4003580 con 0x7f7b0c071200 2024-03-27T08:26:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff891338640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff88c1358c0 0x7ff88c137ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39464/0 (socket says 172.21.15.104:39464) 2024-03-27T08:26:14.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff891338640 1 -- 172.21.15.104:0/3834331850 learned_addr learned my addr 172.21.15.104:0/3834331850 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 >> v2:172.21.15.104:3300/0 conn(0x7f7b0c135860 msgr2=0x7f7b0c137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T08:26:14.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 --2- 172.21.15.104:0/70475475 >> v2:172.21.15.104:3300/0 conn(0x7f7b0c135860 0x7f7b0c137c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 --> v1:172.21.15.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c074ad0 con 0x7f7b0c071200 2024-03-27T08:26:14.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 <== mon.0 v1:172.21.15.104:6789/0 3 ==== mon_map magic: 0 ==== 205+0+0 (unknown 4189522127 0 0) 0x7f7afc0015a0 con 0x7f7b0c071200 2024-03-27T08:26:14.433 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 >> v1:172.21.15.104:6789/0 conn(0x7f7b0c071200 legacy=0x7f7b0c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb93affd640 1 -- 172.21.15.104:0/2745587395 >> v1:172.21.15.104:6789/0 conn(0x7fb93c0706b0 legacy=0x7fb93c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86000a06a0 con 0x7f8600131650 2024-03-27T08:26:14.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 --2- 172.21.15.104:0/70475475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7af4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c072920 con 0x7f7af4003a20 2024-03-27T08:26:14.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f7b0affd640 1 --2- 172.21.15.104:0/70475475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7af4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7fb93affd640 1 -- 172.21.15.104:0/2745587395 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c074b40 con 0x7fb93c12f5e0 2024-03-27T08:26:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f8600132170 con 0x7f86001378c0 2024-03-27T08:26:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7f85fffff640 1 --2- 172.21.15.104:0/2578100967 >> v2:172.21.15.104:3300/0 conn(0x7f86001378c0 0x7f8600139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.435+0000 7ff891338640 1 -- 172.21.15.104:0/3834331850 >> v1:172.21.15.104:6789/0 conn(0x7ff88c12f5e0 legacy=0x7ff88c12f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.436 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7fb93affd640 1 --2- 172.21.15.104:0/2745587395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c12f5e0 0x7fb93c12f9f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4113e84e7fa29b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.437 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7ff891338640 1 -- 172.21.15.104:0/3834331850 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c074b40 con 0x7ff88c1358c0 2024-03-27T08:26:14.437 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f85fffff640 1 -- 172.21.15.104:0/2578100967 >> v1:172.21.15.104:6789/0 conn(0x7f8600131650 legacy=0x7f8600131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.437 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f85fffff640 1 -- 172.21.15.104:0/2578100967 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860006de30 con 0x7f86001378c0 2024-03-27T08:26:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f7b0affd640 1 -- 172.21.15.104:0/70475475 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af4003580 con 0x7f7af4003a20 2024-03-27T08:26:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7fb939ffb640 1 -- 172.21.15.104:0/2745587395 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb92c009e50 con 0x7fb93c12f5e0 2024-03-27T08:26:14.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7ff891338640 1 --2- 172.21.15.104:0/3834331850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c1358c0 0x7ff88c137ce0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53020c3a0adc9fc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7fb939ffb640 1 -- 172.21.15.104:0/2745587395 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb92c00e230 con 0x7fb93c12f5e0 2024-03-27T08:26:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7fb939ffb640 1 -- 172.21.15.104:0/2745587395 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb92c00e4d0 con 0x7fb93c12f5e0 2024-03-27T08:26:14.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f85fffff640 1 --2- 172.21.15.104:0/2578100967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86001378c0 0x7f8600139ce0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2858ef132c0c812d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f7b0affd640 1 --2- 172.21.15.104:0/70475475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7af4003e70 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2208b0d94902cfef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7ff890b37640 1 -- 172.21.15.104:0/3834331850 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff87c009e50 con 0x7ff88c1358c0 2024-03-27T08:26:14.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7ff890b37640 1 -- 172.21.15.104:0/3834331850 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff87c00e230 con 0x7ff88c1358c0 2024-03-27T08:26:14.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7ff890b37640 1 -- 172.21.15.104:0/3834331850 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff87c00e4d0 con 0x7ff88c1358c0 2024-03-27T08:26:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f85ff7fe640 1 -- 172.21.15.104:0/2578100967 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f85ec009d90 con 0x7f86001378c0 2024-03-27T08:26:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7afc007be0 con 0x7f7af4003a20 2024-03-27T08:26:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f85ff7fe640 1 -- 172.21.15.104:0/2578100967 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f85ec00e230 con 0x7f86001378c0 2024-03-27T08:26:14.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f85ff7fe640 1 -- 172.21.15.104:0/2578100967 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f85ec00e4d0 con 0x7f86001378c0 2024-03-27T08:26:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7afc005a60 con 0x7f7af4003a20 2024-03-27T08:26:14.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.436+0000 7f7b09ffb640 1 -- 172.21.15.104:0/70475475 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7afc005d00 con 0x7f7af4003a20 2024-03-27T08:26:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.437+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c12f5e0 msgr2=0x7fb93c12f9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.437+0000 7fb9412ad640 1 --2- 172.21.15.104:0/2745587395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c12f5e0 0x7fb93c12f9f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.437+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 shutdown_connections 2024-03-27T08:26:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.437+0000 7fb9412ad640 1 --2- 172.21.15.104:0/2745587395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c12f5e0 0x7fb93c12f9f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.437+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 >> 172.21.15.104:0/2745587395 conn(0x7fb93c0a2900 msgr2=0x7fb93c0a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.437+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 shutdown_connections 2024-03-27T08:26:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86001378c0 msgr2=0x7f8600139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 --2- 172.21.15.104:0/2578100967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86001378c0 0x7f8600139ce0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.444 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 -- 172.21.15.104:0/3834331850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c1358c0 msgr2=0x7ff88c137ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7fb9412ad640 1 -- 172.21.15.104:0/2745587395 wait complete. 2024-03-27T08:26:14.445 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 --2- 172.21.15.104:0/3834331850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c1358c0 0x7ff88c137ce0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 shutdown_connections 2024-03-27T08:26:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 --2- 172.21.15.104:0/2578100967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86001378c0 0x7f8600139ce0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 >> 172.21.15.104:0/2578100967 conn(0x7f86000a15e0 msgr2=0x7f86000a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 -- 172.21.15.104:0/3834331850 shutdown_connections 2024-03-27T08:26:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 --2- 172.21.15.104:0/3834331850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c1358c0 0x7ff88c137ce0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 -- 172.21.15.104:0/3834331850 >> 172.21.15.104:0/3834331850 conn(0x7ff88c0a2900 msgr2=0x7ff88c0a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 shutdown_connections 2024-03-27T08:26:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 msgr2=0x7f7af4003e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 1 -- 172.21.15.104:0/2578100967 wait complete. 2024-03-27T08:26:14.448 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 --2- 172.21.15.104:0/70475475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7af4003e70 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.448 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7fb9412ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.449 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 -- 172.21.15.104:0/3834331850 shutdown_connections 2024-03-27T08:26:14.449 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 1 -- 172.21.15.104:0/3834331850 wait complete. 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 shutdown_connections 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 --2- 172.21.15.104:0/70475475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7af4003e70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 --2- 172.21.15.104:0/70475475 >> v2:172.21.15.104:3300/0 conn(0x7f7b0c135860 0x7f7b0c137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 >> 172.21.15.104:0/70475475 conn(0x7f7b0c0a0f00 msgr2=0x7f7b0c0a1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 shutdown_connections 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 1 -- 172.21.15.104:0/70475475 wait complete. 2024-03-27T08:26:14.450 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f8606e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.451 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7ff893d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.451 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.438+0000 7f7b1186d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.439+0000 7fb9412ad640 1 Processor -- start 2024-03-27T08:26:14.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.439+0000 7ff893d9d640 1 Processor -- start 2024-03-27T08:26:14.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.440+0000 7f7b1186d640 1 Processor -- start 2024-03-27T08:26:14.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.440+0000 7f8606e2d640 1 Processor -- start 2024-03-27T08:26:14.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.440+0000 7ff893d9d640 1 -- start start 2024-03-27T08:26:14.453 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.440+0000 7fb9412ad640 1 -- start start 2024-03-27T08:26:14.454 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.440+0000 7f8606e2d640 1 -- start start 2024-03-27T08:26:14.454 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.440+0000 7f7b1186d640 1 -- start start 2024-03-27T08:26:14.454 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7fb9412ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 0x7fb93c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7fb9412ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c12d020 con 0x7fb93c0706b0 2024-03-27T08:26:14.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f7b1186d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7b0c1c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f7b1186d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c132770 con 0x7f7af4003a20 2024-03-27T08:26:14.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7fb93affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 0x7fb93c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.455 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f8606e2d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 0x7f86001c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f8606e2d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8600121cc0 con 0x7f8600131650 2024-03-27T08:26:14.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7fb93affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 0x7fb93c06e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39510/0 (socket says 172.21.15.104:39510) 2024-03-27T08:26:14.456 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7fb93affd640 1 -- 172.21.15.104:0/1888748662 learned_addr learned my addr 172.21.15.104:0/1888748662 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff893d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 0x7ff88c1c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff893d9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c11d1e0 con 0x7ff88c12f5e0 2024-03-27T08:26:14.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff891b39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 0x7ff88c1c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f8604bc9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 0x7f86001c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.457 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff891b39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 0x7ff88c1c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39514/0 (socket says 172.21.15.104:39514) 2024-03-27T08:26:14.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff891b39640 1 -- 172.21.15.104:0/1310543281 learned_addr learned my addr 172.21.15.104:0/1310543281 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f7b0affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7b0c1c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f7b0affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7b0c1c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39526/0 (socket says 172.21.15.104:39526) 2024-03-27T08:26:14.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f7b0affd640 1 -- 172.21.15.104:0/940962459 learned_addr learned my addr 172.21.15.104:0/940962459 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f8604bc9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 0x7f86001c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39532/0 (socket says 172.21.15.104:39532) 2024-03-27T08:26:14.458 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7f8604bc9640 1 -- 172.21.15.104:0/1142728601 learned_addr learned my addr 172.21.15.104:0/1142728601 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff891b39640 1 -- 172.21.15.104:0/1310543281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c072800 con 0x7ff88c12f5e0 2024-03-27T08:26:14.459 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7fb93affd640 1 -- 172.21.15.104:0/1888748662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c072800 con 0x7fb93c0706b0 2024-03-27T08:26:14.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.441+0000 7ff891b39640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 0x7ff88c1c4210 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff88800c870 con 0x7ff88c12f5e0 2024-03-27T08:26:14.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f7b0affd640 1 -- 172.21.15.104:0/940962459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c1304e0 con 0x7f7af4003a20 2024-03-27T08:26:14.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff88800c9d0 con 0x7ff88c12f5e0 2024-03-27T08:26:14.460 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7fb93affd640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 0x7fb93c06e4d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff88800cc70 con 0x7ff88c12f5e0 2024-03-27T08:26:14.461 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f8604bc9640 1 -- 172.21.15.104:0/1142728601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600120950 con 0x7f8600131650 2024-03-27T08:26:14.462 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c11eb40 con 0x7ff88c12f5e0 2024-03-27T08:26:14.462 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c11d470 con 0x7ff88c12f5e0 2024-03-27T08:26:14.462 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f7b0affd640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7b0c1c4d70 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.462 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f8604bc9640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 0x7f86001c6290 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.463 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb92c00ed50 con 0x7fb93c0706b0 2024-03-27T08:26:14.463 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c09d2f0 con 0x7fb93c0706b0 2024-03-27T08:26:14.463 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c070130 con 0x7fb93c0706b0 2024-03-27T08:26:14.464 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f85f4005a60 con 0x7f8600131650 2024-03-27T08:26:14.464 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7afc005bd0 con 0x7f7af4003a20 2024-03-27T08:26:14.464 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.442+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86001c5be0 con 0x7f8600131650 2024-03-27T08:26:14.464 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001200f0 con 0x7f8600131650 2024-03-27T08:26:14.465 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb92c00eeb0 con 0x7fb93c0706b0 2024-03-27T08:26:14.465 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff88c1c4010 con 0x7ff88c12f5e0 2024-03-27T08:26:14.465 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c073c30 con 0x7f7af4003a20 2024-03-27T08:26:14.465 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c131b80 con 0x7f7af4003a20 2024-03-27T08:26:14.466 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff888011310 con 0x7ff88c12f5e0 2024-03-27T08:26:14.466 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f85f4005bc0 con 0x7f8600131650 2024-03-27T08:26:14.466 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7ff8827fc640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff8700387a0 0x7ff87003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f85f4005e60 con 0x7f8600131650 2024-03-27T08:26:14.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff8880486c0 con 0x7ff88c12f5e0 2024-03-27T08:26:14.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7ff8827fc640 1 --2- 172.21.15.104:0/1310543281 >> v2:172.21.15.177:6805/2552358937 conn(0x7ff87003d7a0 0x7ff87003fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 --> v2:172.21.15.177:6805/2552358937 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8700402c0 con 0x7ff87003d7a0 2024-03-27T08:26:14.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7ff88800ce20 con 0x7ff88c12f5e0 2024-03-27T08:26:14.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb92c015420 con 0x7fb93c0706b0 2024-03-27T08:26:14.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7afc001430 con 0x7f7af4003a20 2024-03-27T08:26:14.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7afc0015a0 con 0x7f7af4003a20 2024-03-27T08:26:14.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.443+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb91c000f80 con 0x7fb93c0706b0 2024-03-27T08:26:14.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fb92c015580 con 0x7fb93c0706b0 2024-03-27T08:26:14.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7fb9237fe640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb9280387d0 0x7fb92803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7aec000f80 con 0x7f7af4003a20 2024-03-27T08:26:14.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85e0000f80 con 0x7f8600131650 2024-03-27T08:26:14.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fb92c04ab20 con 0x7fb93c0706b0 2024-03-27T08:26:14.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7fb9237fe640 1 --2- 172.21.15.104:0/1888748662 >> v2:172.21.15.104:6801/1224298329 conn(0x7fb92803d7d0 0x7fb92803fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7ff891338640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff8700387a0 0x7ff87003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 --> v2:172.21.15.104:6801/1224298329 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9280402f0 con 0x7fb92803d7d0 2024-03-27T08:26:14.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.444+0000 7fb93a7fc640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb9280387d0 0x7fb92803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f85f400c780 con 0x7f8600131650 2024-03-27T08:26:14.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7fb92c0143b0 con 0x7fb93c0706b0 2024-03-27T08:26:14.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7afc007000 con 0x7f7af4003a20 2024-03-27T08:26:14.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7ff89233a640 1 --2- 172.21.15.104:0/1310543281 >> v2:172.21.15.177:6805/2552358937 conn(0x7ff87003d7a0 0x7ff87003fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:26:14.473 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f7af3fff640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7af80387a0 0x7f7af803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.473 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f7afc049090 con 0x7f7af4003a20 2024-03-27T08:26:14.473 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f85fd7fa640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f85e40387a0 0x7f85e403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.474 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f7af3fff640 1 --2- 172.21.15.104:0/940962459 >> v2:172.21.15.177:6807/929709834 conn(0x7f7af803d7a0 0x7f7af803fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.474 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 --> v2:172.21.15.177:6807/929709834 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7af80402c0 con 0x7f7af803d7a0 2024-03-27T08:26:14.474 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f7afc0493b0 con 0x7f7af4003a20 2024-03-27T08:26:14.474 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.445+0000 7fb93a7fc640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb9280387d0 0x7fb92803ace0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.475 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f85f4048260 con 0x7f8600131650 2024-03-27T08:26:14.475 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f7b0a7fc640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7af80387a0 0x7f7af803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f85fd7fa640 1 --2- 172.21.15.104:0/1142728601 >> v2:172.21.15.104:6804/1714017423 conn(0x7f85e403d7a0 0x7f85e403fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 --> v2:172.21.15.104:6804/1714017423 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85e40402c0 con 0x7f85e403d7a0 2024-03-27T08:26:14.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f85fffff640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f85e40387a0 0x7f85e403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f85f4048640 con 0x7f8600131650 2024-03-27T08:26:14.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7ff891338640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff8700387a0 0x7ff87003acb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7fb93b7fe640 1 --2- 172.21.15.104:0/1888748662 >> v2:172.21.15.104:6801/1224298329 conn(0x7fb92803d7d0 0x7fb92803fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:26:14.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f7b0b7fe640 1 --2- 172.21.15.104:0/940962459 >> v2:172.21.15.177:6807/929709834 conn(0x7f7af803d7a0 0x7f7af803fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:26:14.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.446+0000 7f85fffff640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f85e40387a0 0x7f85e403acb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.478 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.447+0000 7f7b0a7fc640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7af80387a0 0x7f7af803acb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.478 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.447+0000 7f86053ca640 1 --2- 172.21.15.104:0/1142728601 >> v2:172.21.15.104:6804/1714017423 conn(0x7f85e403d7a0 0x7f85e403fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:26:14.478 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.450+0000 7f7b0b7fe640 1 --2- 172.21.15.104:0/940962459 >> v2:172.21.15.177:6807/929709834 conn(0x7f7af803d7a0 0x7f7af803fc30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.479 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.450+0000 7ff89233a640 1 --2- 172.21.15.104:0/1310543281 >> v2:172.21.15.177:6805/2552358937 conn(0x7ff87003d7a0 0x7ff87003fc30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.479 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.451+0000 7fb93b7fe640 1 --2- 172.21.15.104:0/1888748662 >> v2:172.21.15.104:6801/1224298329 conn(0x7fb92803d7d0 0x7fb92803fc60 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.479 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.451+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f7af80402c0 con 0x7f7af803d7a0 2024-03-27T08:26:14.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.451+0000 7f86053ca640 1 --2- 172.21.15.104:0/1142728601 >> v2:172.21.15.104:6804/1714017423 conn(0x7f85e403d7a0 0x7f85e403fc30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.452+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== osd.2 v2:172.21.15.177:6805/2552358937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7ff8700402c0 con 0x7ff87003d7a0 2024-03-27T08:26:14.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.454+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7fb9280402f0 con 0x7fb92803d7d0 2024-03-27T08:26:14.481 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.454+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f85e40402c0 con 0x7f85e403d7a0 2024-03-27T08:26:14.481 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.456+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 --> v2:172.21.15.177:6807/929709834 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7aec002bb0 con 0x7f7af803d7a0 2024-03-27T08:26:14.481 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.456+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 --> v2:172.21.15.177:6805/2552358937 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff88c135b10 con 0x7ff87003d7a0 2024-03-27T08:26:14.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.457+0000 7f7af3fff640 1 -- 172.21.15.104:0/940962459 <== osd.3 v2:172.21.15.177:6807/929709834 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7aec002bb0 con 0x7f7af803d7a0 2024-03-27T08:26:14.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.457+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 --> v2:172.21.15.104:6801/1224298329 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb91c002c30 con 0x7fb92803d7d0 2024-03-27T08:26:14.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.457+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 >> v2:172.21.15.177:6807/929709834 conn(0x7f7af803d7a0 msgr2=0x7f7af803fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.457+0000 7f7b1186d640 1 --2- 172.21.15.104:0/940962459 >> v2:172.21.15.177:6807/929709834 conn(0x7f7af803d7a0 0x7f7af803fc30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.457+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7af80387a0 msgr2=0x7f7af803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.457+0000 7f7b1186d640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7af80387a0 0x7f7af803acb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 msgr2=0x7f7b0c1c4d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7b0c1c4d70 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff8827fc640 1 -- 172.21.15.104:0/1310543281 <== osd.2 v2:172.21.15.177:6805/2552358937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff88c135b10 con 0x7ff87003d7a0 2024-03-27T08:26:14.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 shutdown_connections 2024-03-27T08:26:14.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 --2- 172.21.15.104:0/940962459 >> v2:172.21.15.177:6807/929709834 conn(0x7f7af803d7a0 0x7f7af803fc30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7af80387a0 0x7f7af803acb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 --2- 172.21.15.104:0/940962459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7af4003a20 0x7f7b0c1c4d70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 >> 172.21.15.104:0/940962459 conn(0x7f7b0c0a0f00 msgr2=0x7f7b0c097410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 --> v2:172.21.15.104:6804/1714017423 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85e0002bb0 con 0x7f85e403d7a0 2024-03-27T08:26:14.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 shutdown_connections 2024-03-27T08:26:14.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7f7b1186d640 1 -- 172.21.15.104:0/940962459 wait complete. 2024-03-27T08:26:14.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 >> v2:172.21.15.177:6805/2552358937 conn(0x7ff87003d7a0 msgr2=0x7ff87003fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 --2- 172.21.15.104:0/1310543281 >> v2:172.21.15.177:6805/2552358937 conn(0x7ff87003d7a0 0x7ff87003fc30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff8700387a0 msgr2=0x7ff87003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff8700387a0 0x7ff87003acb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 msgr2=0x7ff88c1c4210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 0x7ff88c1c4210 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 shutdown_connections 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 --2- 172.21.15.104:0/1310543281 >> v2:172.21.15.177:6805/2552358937 conn(0x7ff87003d7a0 0x7ff87003fc30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff8700387a0 0x7ff87003acb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 --2- 172.21.15.104:0/1310543281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff88c12f5e0 0x7ff88c1c4210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 >> 172.21.15.104:0/1310543281 conn(0x7ff88c0a2900 msgr2=0x7ff88c1ba9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 shutdown_connections 2024-03-27T08:26:14.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.458+0000 7ff893d9d640 1 -- 172.21.15.104:0/1310543281 wait complete. 2024-03-27T08:26:14.490 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.459+0000 7fb9237fe640 1 -- 172.21.15.104:0/1888748662 <== osd.1 v2:172.21.15.104:6801/1224298329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb91c002c30 con 0x7fb92803d7d0 2024-03-27T08:26:14.490 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7f85fd7fa640 1 -- 172.21.15.104:0/1142728601 <== osd.0 v2:172.21.15.104:6804/1714017423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f85e0002bb0 con 0x7f85e403d7a0 2024-03-27T08:26:14.492 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 >> v2:172.21.15.104:6801/1224298329 conn(0x7fb92803d7d0 msgr2=0x7fb92803fc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.492 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 --2- 172.21.15.104:0/1888748662 >> v2:172.21.15.104:6801/1224298329 conn(0x7fb92803d7d0 0x7fb92803fc60 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.492 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb9280387d0 msgr2=0x7fb92803ace0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.492 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb9280387d0 0x7fb92803ace0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 msgr2=0x7fb93c06e4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 0x7fb93c06e4d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 shutdown_connections 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb9280387d0 0x7fb92803ace0 crc :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 --2- 172.21.15.104:0/1888748662 >> v2:172.21.15.104:6801/1224298329 conn(0x7fb92803d7d0 0x7fb92803fc60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 --2- 172.21.15.104:0/1888748662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c0706b0 0x7fb93c06e4d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.493 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 >> 172.21.15.104:0/1888748662 conn(0x7fb93c0a2900 msgr2=0x7fb93c1ba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.494 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 >> v2:172.21.15.104:6804/1714017423 conn(0x7f85e403d7a0 msgr2=0x7f85e403fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.494 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7f8606e2d640 1 --2- 172.21.15.104:0/1142728601 >> v2:172.21.15.104:6804/1714017423 conn(0x7f85e403d7a0 0x7f85e403fc30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.494 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 shutdown_connections 2024-03-27T08:26:14.494 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.460+0000 7fb9412ad640 1 -- 172.21.15.104:0/1888748662 wait complete. 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f85e40387a0 msgr2=0x7f85e403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f85e40387a0 0x7f85e403acb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 msgr2=0x7f86001c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 0x7f86001c6290 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 shutdown_connections 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 --2- 172.21.15.104:0/1142728601 >> v2:172.21.15.104:6804/1714017423 conn(0x7f85e403d7a0 0x7f85e403fc30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.495 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f85e40387a0 0x7f85e403acb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.496 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 --2- 172.21.15.104:0/1142728601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8600131650 0x7f86001c6290 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.496 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 >> 172.21.15.104:0/1142728601 conn(0x7f86000a15e0 msgr2=0x7f86000a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.496 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 shutdown_connections 2024-03-27T08:26:14.496 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.461+0000 7f8606e2d640 1 -- 172.21.15.104:0/1142728601 wait complete. 2024-03-27T08:26:14.496 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:14.497 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:14.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-27T08:26:14.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-27T08:26:14.499 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:14.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-27T08:26:14.500 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:14.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-27T08:26:14.655 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.658+0000 7ff69af9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.655 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.658+0000 7f953d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.656 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.658+0000 7f9093a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.656 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.664+0000 7f953d46d640 1 Processor -- start 2024-03-27T08:26:14.656 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.664+0000 7f953d46d640 1 -- start start 2024-03-27T08:26:14.656 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.664+0000 7ff69af9d640 1 Processor -- start 2024-03-27T08:26:14.657 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.664+0000 7f9093a6d640 1 Processor -- start 2024-03-27T08:26:14.657 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.664+0000 7ff69af9d640 1 -- start start 2024-03-27T08:26:14.657 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.664+0000 7f9093a6d640 1 -- start start 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f953d46d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f953809b550 0x7f95381362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f953d46d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953809be00 con 0x7f953809aa30 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f953d46d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f953806c760 con 0x7f953809b550 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f95367fc640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f953809b550 0x7f95381362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9536ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f953809aa30 0x7f953809ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49668/0 (socket says 172.21.15.104:49668) 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f95367fc640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f953809b550 0x7f95381362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39546/0 (socket says 172.21.15.104:39546) 2024-03-27T08:26:14.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9536ffd640 1 -- 172.21.15.104:0/506924246 learned_addr learned my addr 172.21.15.104:0/506924246 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:14.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9093a6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f908c12f5e0 0x7f908c12f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f95367fc640 1 -- 172.21.15.104:0/506924246 >> v1:172.21.15.104:6789/0 conn(0x7f953809aa30 legacy=0x7f953809ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9093a6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f908c073520 con 0x7f908c0706b0 2024-03-27T08:26:14.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9093a6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f908c071880 con 0x7f908c12f5e0 2024-03-27T08:26:14.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9091809640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f908c12f5e0 0x7f908c12f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f95367fc640 1 -- 172.21.15.104:0/506924246 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953806e940 con 0x7f953809b550 2024-03-27T08:26:14.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9091008640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f908c0706b0 0x7f908c070ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49674/0 (socket says 172.21.15.104:49674) 2024-03-27T08:26:14.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7f9091008640 1 -- 172.21.15.104:0/3463661842 learned_addr learned my addr 172.21.15.104:0/3463661842 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:14.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.665+0000 7ff69af9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff694131830 0x7ff694131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f95367fc640 1 --2- 172.21.15.104:0/506924246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809b550 0x7f95381362b0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d27e4903788946 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff69af9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6940735e0 con 0x7ff694134580 2024-03-27T08:26:14.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff69af9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7ff6940a0260 con 0x7ff694131830 2024-03-27T08:26:14.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9090807640 1 -- 172.21.15.104:0/3463661842 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402559297 0 0) 0x7f908c073520 con 0x7f908c0706b0 2024-03-27T08:26:14.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9090807640 1 -- 172.21.15.104:0/3463661842 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9078003580 con 0x7f908c0706b0 2024-03-27T08:26:14.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff693fff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff694134580 0x7ff694134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49682/0 (socket says 172.21.15.104:49682) 2024-03-27T08:26:14.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9091809640 1 -- 172.21.15.104:0/3463661842 >> v1:172.21.15.104:6789/0 conn(0x7f908c0706b0 legacy=0x7f908c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff693fff640 1 -- 172.21.15.104:0/2882536538 learned_addr learned my addr 172.21.15.104:0/2882536538 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:14.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff698d39640 1 --2- 172.21.15.104:0/2882536538 >> v2:172.21.15.104:3300/0 conn(0x7ff694131830 0x7ff694131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9091809640 1 -- 172.21.15.104:0/3463661842 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c074b40 con 0x7f908c12f5e0 2024-03-27T08:26:14.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9535ffb640 1 -- 172.21.15.104:0/506924246 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f952c009c80 con 0x7f953809b550 2024-03-27T08:26:14.665 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9535ffb640 1 -- 172.21.15.104:0/506924246 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f952c009de0 con 0x7f953809b550 2024-03-27T08:26:14.665 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9535ffb640 1 -- 172.21.15.104:0/506924246 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f952c00e360 con 0x7f953809b550 2024-03-27T08:26:14.665 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9091809640 1 --2- 172.21.15.104:0/3463661842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c12f5e0 0x7f908c12f9f0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0d7e424d83ecc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.666 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff698d39640 1 -- 172.21.15.104:0/2882536538 >> v1:172.21.15.104:6789/0 conn(0x7ff694134580 legacy=0x7ff694134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.666 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff698d39640 1 -- 172.21.15.104:0/2882536538 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69406eb80 con 0x7ff694131830 2024-03-27T08:26:14.666 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9090807640 1 -- 172.21.15.104:0/3463661842 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9088009d90 con 0x7f908c12f5e0 2024-03-27T08:26:14.666 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7ff698d39640 1 --2- 172.21.15.104:0/2882536538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694131c40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40e42cab84fa381 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9090807640 1 -- 172.21.15.104:0/3463661842 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f908800e230 con 0x7f908c12f5e0 2024-03-27T08:26:14.667 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.666+0000 7f9090807640 1 -- 172.21.15.104:0/3463661842 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f908800e4d0 con 0x7f908c12f5e0 2024-03-27T08:26:14.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7ff6937fe640 1 -- 172.21.15.104:0/2882536538 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff688009d90 con 0x7ff694131830 2024-03-27T08:26:14.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7ff6937fe640 1 -- 172.21.15.104:0/2882536538 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff68800e230 con 0x7ff694131830 2024-03-27T08:26:14.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7ff6937fe640 1 -- 172.21.15.104:0/2882536538 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff68800e4d0 con 0x7ff694131830 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 -- 172.21.15.104:0/506924246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809b550 msgr2=0x7f95381362b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 --2- 172.21.15.104:0/506924246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809b550 0x7f95381362b0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 -- 172.21.15.104:0/506924246 shutdown_connections 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 --2- 172.21.15.104:0/506924246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809b550 0x7f95381362b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 -- 172.21.15.104:0/506924246 >> 172.21.15.104:0/506924246 conn(0x7f95380a0ba0 msgr2=0x7f95380a0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 -- 172.21.15.104:0/506924246 shutdown_connections 2024-03-27T08:26:14.669 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.667+0000 7f953d46d640 1 -- 172.21.15.104:0/506924246 wait complete. 2024-03-27T08:26:14.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 -- 172.21.15.104:0/3463661842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c12f5e0 msgr2=0x7f908c12f9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 --2- 172.21.15.104:0/3463661842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c12f5e0 0x7f908c12f9f0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 -- 172.21.15.104:0/3463661842 shutdown_connections 2024-03-27T08:26:14.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f953d46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 --2- 172.21.15.104:0/3463661842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c12f5e0 0x7f908c12f9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 -- 172.21.15.104:0/3463661842 >> 172.21.15.104:0/3463661842 conn(0x7f908c0a2900 msgr2=0x7f908c0a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.671 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 -- 172.21.15.104:0/2882536538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 msgr2=0x7ff694131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 --2- 172.21.15.104:0/2882536538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694131c40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 -- 172.21.15.104:0/3463661842 shutdown_connections 2024-03-27T08:26:14.672 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 1 -- 172.21.15.104:0/3463661842 wait complete. 2024-03-27T08:26:14.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 -- 172.21.15.104:0/2882536538 shutdown_connections 2024-03-27T08:26:14.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 --2- 172.21.15.104:0/2882536538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694131c40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 -- 172.21.15.104:0/2882536538 >> 172.21.15.104:0/2882536538 conn(0x7ff6940a4ae0 msgr2=0x7ff6940a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 -- 172.21.15.104:0/2882536538 shutdown_connections 2024-03-27T08:26:14.673 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 1 -- 172.21.15.104:0/2882536538 wait complete. 2024-03-27T08:26:14.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7f9093a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.668+0000 7ff69af9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.669+0000 7f953d46d640 1 Processor -- start 2024-03-27T08:26:14.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.669+0000 7f953d46d640 1 -- start start 2024-03-27T08:26:14.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.670+0000 7f9093a6d640 1 Processor -- start 2024-03-27T08:26:14.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.670+0000 7f9093a6d640 1 -- start start 2024-03-27T08:26:14.676 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.670+0000 7ff69af9d640 1 Processor -- start 2024-03-27T08:26:14.676 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.670+0000 7ff69af9d640 1 -- start start 2024-03-27T08:26:14.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.670+0000 7f953d46d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 0x7f953808cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f953d46d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9538090a60 con 0x7f953809aa30 2024-03-27T08:26:14.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 0x7f953808cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9093a6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 0x7f908c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9093a6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f908c12f440 con 0x7f908c0706b0 2024-03-27T08:26:14.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 0x7f953808cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39574/0 (socket says 172.21.15.104:39574) 2024-03-27T08:26:14.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9536ffd640 1 -- 172.21.15.104:0/3064150753 learned_addr learned my addr 172.21.15.104:0/3064150753 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9091809640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 0x7f908c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9091809640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 0x7f908c09aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39584/0 (socket says 172.21.15.104:39584) 2024-03-27T08:26:14.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9091809640 1 -- 172.21.15.104:0/3361714583 learned_addr learned my addr 172.21.15.104:0/3361714583 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9536ffd640 1 -- 172.21.15.104:0/3064150753 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9538099d20 con 0x7f953809aa30 2024-03-27T08:26:14.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7ff69af9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7ff69af9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff694071fd0 con 0x7ff694131830 2024-03-27T08:26:14.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.671+0000 7f9536ffd640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 0x7f953808cc30 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7ff698d39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7ff698d39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39590/0 (socket says 172.21.15.104:39590) 2024-03-27T08:26:14.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7ff698d39640 1 -- 172.21.15.104:0/574730347 learned_addr learned my addr 172.21.15.104:0/574730347 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.682 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f9091809640 1 -- 172.21.15.104:0/3361714583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c072800 con 0x7f908c0706b0 2024-03-27T08:26:14.682 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9528004b80 con 0x7f953809aa30 2024-03-27T08:26:14.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9528004ce0 con 0x7f953809aa30 2024-03-27T08:26:14.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9538090c70 con 0x7f953809aa30 2024-03-27T08:26:14.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9528007120 con 0x7f953809aa30 2024-03-27T08:26:14.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f9091809640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 0x7f908c09aa30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9538090200 con 0x7f953809aa30 2024-03-27T08:26:14.684 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7ff698d39640 1 -- 172.21.15.104:0/574730347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69412f270 con 0x7ff694131830 2024-03-27T08:26:14.684 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9088014540 con 0x7f908c0706b0 2024-03-27T08:26:14.684 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f90880146a0 con 0x7f908c0706b0 2024-03-27T08:26:14.684 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.672+0000 7ff698d39640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694070b80 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.685 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9088014940 con 0x7f908c0706b0 2024-03-27T08:26:14.685 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c09e090 con 0x7f908c0706b0 2024-03-27T08:26:14.685 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c09c230 con 0x7f908c0706b0 2024-03-27T08:26:14.685 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9538063450 con 0x7f953809aa30 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff688014620 con 0x7ff694131830 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff688014780 con 0x7ff694131830 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff688014a20 con 0x7ff694131830 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.673+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69411d650 con 0x7ff694131830 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.674+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69411eeb0 con 0x7ff694131830 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.674+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f9528007320 con 0x7f953809aa30 2024-03-27T08:26:14.686 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.674+0000 7f951f7fe640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9524038750 0x7f952403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.674+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f95280482b0 con 0x7f953809aa30 2024-03-27T08:26:14.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.674+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f9088014aa0 con 0x7f908c0706b0 2024-03-27T08:26:14.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.674+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c074e60 con 0x7f908c0706b0 2024-03-27T08:26:14.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7f90827fc640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9070038840 0x7f907003ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f9088019040 con 0x7f908c0706b0 2024-03-27T08:26:14.688 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7f95367fc640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9524038750 0x7f952403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.688 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff694063450 con 0x7ff694131830 2024-03-27T08:26:14.688 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7f9091008640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9070038840 0x7f907003ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.688 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7f95367fc640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9524038750 0x7f952403ac60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.675+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff688014b80 con 0x7ff694131830 2024-03-27T08:26:14.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.676+0000 7ff6917fa640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff6780387f0 0x7ff67803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.676+0000 7ff693fff640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff6780387f0 0x7ff67803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.676+0000 7f9091008640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9070038840 0x7f907003ad50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.676+0000 7ff693fff640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff6780387f0 0x7ff67803ad00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.681+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff688019040 con 0x7ff694131830 2024-03-27T08:26:14.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.681+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f952800aba0 con 0x7f953809aa30 2024-03-27T08:26:14.691 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.686+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f908800fbc0 con 0x7f908c0706b0 2024-03-27T08:26:14.691 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.695+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff688011380 con 0x7ff694131830 2024-03-27T08:26:14.702 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.704+0000 7f484ac2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.710+0000 7f484ac2d640 1 Processor -- start 2024-03-27T08:26:14.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.711+0000 7f484ac2d640 1 -- start start 2024-03-27T08:26:14.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.712+0000 7f484ac2d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f48441378c0 0x7f4844139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.712+0000 7f484ac2d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48440a06a0 con 0x7f4844131650 2024-03-27T08:26:14.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.712+0000 7f484ac2d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f4844132170 con 0x7f48441378c0 2024-03-27T08:26:14.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f4843fff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f48441378c0 0x7f4844139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f48489c9640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4844131650 0x7f4844131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:49698/0 (socket says 172.21.15.104:49698) 2024-03-27T08:26:14.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f48489c9640 1 -- 172.21.15.104:0/2342204670 learned_addr learned my addr 172.21.15.104:0/2342204670 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f4843fff640 1 -- 172.21.15.104:0/2342204670 >> v1:172.21.15.104:6789/0 conn(0x7f4844131650 legacy=0x7f4844131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f4843fff640 1 -- 172.21.15.104:0/2342204670 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484406de30 con 0x7f48441378c0 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f4843fff640 1 --2- 172.21.15.104:0/2342204670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48441378c0 0x7f4844139ce0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e0aefc568199dc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f48437fe640 1 -- 172.21.15.104:0/2342204670 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4830009d90 con 0x7f48441378c0 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.713+0000 7f48437fe640 1 -- 172.21.15.104:0/2342204670 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f483000e230 con 0x7f48441378c0 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f48437fe640 1 -- 172.21.15.104:0/2342204670 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f483000e4d0 con 0x7f48441378c0 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f484ac2d640 1 -- 172.21.15.104:0/2342204670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48441378c0 msgr2=0x7f4844139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f484ac2d640 1 --2- 172.21.15.104:0/2342204670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48441378c0 0x7f4844139ce0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.707 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f484ac2d640 1 -- 172.21.15.104:0/2342204670 shutdown_connections 2024-03-27T08:26:14.708 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f484ac2d640 1 --2- 172.21.15.104:0/2342204670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48441378c0 0x7f4844139ce0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.708 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f484ac2d640 1 -- 172.21.15.104:0/2342204670 >> 172.21.15.104:0/2342204670 conn(0x7f48440a15e0 msgr2=0x7f48440a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.708 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.714+0000 7f484ac2d640 1 -- 172.21.15.104:0/2342204670 shutdown_connections 2024-03-27T08:26:14.708 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.715+0000 7f484ac2d640 1 -- 172.21.15.104:0/2342204670 wait complete. 2024-03-27T08:26:14.708 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.715+0000 7f484ac2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:14.709 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.717+0000 7f484ac2d640 1 Processor -- start 2024-03-27T08:26:14.709 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.717+0000 7f484ac2d640 1 -- start start 2024-03-27T08:26:14.711 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.719+0000 7f484ac2d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 0x7f48441c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.711 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.719+0000 7f484ac2d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4844121cc0 con 0x7f4844131650 2024-03-27T08:26:14.712 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.719+0000 7f48489c9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 0x7f48441c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.712 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.719+0000 7f48489c9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 0x7f48441c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39602/0 (socket says 172.21.15.104:39602) 2024-03-27T08:26:14.712 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.719+0000 7f48489c9640 1 -- 172.21.15.104:0/1693139560 learned_addr learned my addr 172.21.15.104:0/1693139560 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f48489c9640 1 -- 172.21.15.104:0/1693139560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4844120950 con 0x7f4844131650 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f48489c9640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 0x7f48441c6290 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4838005a60 con 0x7f4844131650 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48441c5be0 con 0x7f4844131650 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4838005bc0 con 0x7f4844131650 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48441200f0 con 0x7f4844131650 2024-03-27T08:26:14.713 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.720+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4838005a60 con 0x7f4844131650 2024-03-27T08:26:14.714 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.721+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f483800c780 con 0x7f4844131650 2024-03-27T08:26:14.714 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.721+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48441c6090 con 0x7f4844131650 2024-03-27T08:26:14.714 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.722+0000 7f48417fa640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f48280387d0 0x7f482803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:14.714 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.722+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f483800d330 con 0x7f4844131650 2024-03-27T08:26:14.714 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.722+0000 7f4843fff640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f48280387d0 0x7f482803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:14.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.722+0000 7f4843fff640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f48280387d0 0x7f482803ace0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:14.725 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.733+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f4838007770 con 0x7f4844131650 2024-03-27T08:26:14.862 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.869+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f95380a29e0 con 0x7f953809aa30 2024-03-27T08:26:14.862 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.870+0000 7f951f7fe640 1 -- 172.21.15.104:0/3064150753 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f952800a0c0 con 0x7f953809aa30 2024-03-27T08:26:14.862 INFO:teuthology.orchestra.run.smithi104.stdout:38654705669 2024-03-27T08:26:14.864 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9524038750 msgr2=0x7f952403ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.864 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9524038750 0x7f952403ac60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.864 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 msgr2=0x7f953808cc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 0x7f953808cc30 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 shutdown_connections 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9524038750 0x7f952403ac60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 --2- 172.21.15.104:0/3064150753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f953809aa30 0x7f953808cc30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 >> 172.21.15.104:0/3064150753 conn(0x7f95380a0ba0 msgr2=0x7f9538071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 shutdown_connections 2024-03-27T08:26:14.865 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.872+0000 7f953d46d640 1 -- 172.21.15.104:0/3064150753 wait complete. 2024-03-27T08:26:14.873 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.881+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f908c12ff00 con 0x7f908c0706b0 2024-03-27T08:26:14.873 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.881+0000 7f90827fc640 1 -- 172.21.15.104:0/3361714583 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f9088001d40 con 0x7f908c0706b0 2024-03-27T08:26:14.874 INFO:teuthology.orchestra.run.smithi104.stdout:38654705669 2024-03-27T08:26:14.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-03-27T08:26:14.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9070038840 msgr2=0x7f907003ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.876 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9070038840 0x7f907003ad50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 msgr2=0x7f908c09aa30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 0x7f908c09aa30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 shutdown_connections 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f9070038840 0x7f907003ad50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 --2- 172.21.15.104:0/3361714583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f908c0706b0 0x7f908c09aa30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 >> 172.21.15.104:0/3361714583 conn(0x7f908c0a2900 msgr2=0x7f908c06efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 shutdown_connections 2024-03-27T08:26:14.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.883+0000 7f9093a6d640 1 -- 172.21.15.104:0/3361714583 wait complete. 2024-03-27T08:26:14.886 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.894+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff694068a10 con 0x7ff694131830 2024-03-27T08:26:14.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.3 2024-03-27T08:26:14.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.894+0000 7ff6917fa640 1 -- 172.21.15.104:0/574730347 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7ff688010310 con 0x7ff694131830 2024-03-27T08:26:14.887 INFO:teuthology.orchestra.run.smithi104.stdout:38654705669 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff6780387f0 msgr2=0x7ff67803ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff6780387f0 0x7ff67803ad00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 msgr2=0x7ff694070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694070b80 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 shutdown_connections 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff6780387f0 0x7ff67803ad00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 --2- 172.21.15.104:0/574730347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff694131830 0x7ff694070b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 >> 172.21.15.104:0/574730347 conn(0x7ff6940a4ae0 msgr2=0x7ff6941bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 shutdown_connections 2024-03-27T08:26:14.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.896+0000 7ff69af9d640 1 -- 172.21.15.104:0/574730347 wait complete. 2024-03-27T08:26:14.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-03-27T08:26:14.917 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.924+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f484411f190 con 0x7f4844131650 2024-03-27T08:26:14.917 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.925+0000 7f48417fa640 1 -- 172.21.15.104:0/1693139560 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f4838007770 con 0x7f4844131650 2024-03-27T08:26:14.917 INFO:teuthology.orchestra.run.smithi104.stdout:38654705669 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f48280387d0 msgr2=0x7f482803ace0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f48280387d0 0x7f482803ace0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 msgr2=0x7f48441c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 0x7f48441c6290 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 shutdown_connections 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f48280387d0 0x7f482803ace0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 --2- 172.21.15.104:0/1693139560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4844131650 0x7f48441c6290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 >> 172.21.15.104:0/1693139560 conn(0x7f48440a15e0 msgr2=0x7f48440a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 shutdown_connections 2024-03-27T08:26:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:14.927+0000 7f484ac2d640 1 -- 172.21.15.104:0/1693139560 wait complete. 2024-03-27T08:26:14.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-03-27T08:26:15.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-27T08:26:15.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-27T08:26:15.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-27T08:26:15.931 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-27T08:26:16.008 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.009+0000 7fdae0fcf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.009 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.017+0000 7fdae0fcf640 1 Processor -- start 2024-03-27T08:26:16.009 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.017+0000 7fdae0fcf640 1 -- start start 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdae0fcf640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdadc137890 0x7fdadc139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdae0fcf640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdadc0735e0 con 0x7fdadc131830 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdae0fcf640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fdadc0a0260 con 0x7fdadc137890 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdad3fff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdadc137890 0x7fdadc139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdad3fff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdadc137890 0x7fdadc139cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52518/0 (socket says 172.21.15.104:52518) 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdad3fff640 1 -- 172.21.15.104:0/2403547562 learned_addr learned my addr 172.21.15.104:0/2403547562 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.018+0000 7fdada59c640 1 --1- 172.21.15.104:0/2403547562 >> v1:172.21.15.104:6789/0 conn(0x7fdadc131830 0x7fdadc131c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40944/0 (socket says 172.21.15.104:40944) 2024-03-27T08:26:16.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad3fff640 1 -- 172.21.15.104:0/2403547562 >> v1:172.21.15.104:6789/0 conn(0x7fdadc131830 legacy=0x7fdadc131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad9d9b640 1 -- 172.21.15.104:0/2403547562 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788104009 0 0) 0x7fdadc0735e0 con 0x7fdadc131830 2024-03-27T08:26:16.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad3fff640 1 -- 172.21.15.104:0/2403547562 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc06eb80 con 0x7fdadc137890 2024-03-27T08:26:16.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad3fff640 1 --2- 172.21.15.104:0/2403547562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc137890 0x7fdadc139cb0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3014cdf08b69a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad9d9b640 1 -- 172.21.15.104:0/2403547562 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdac4009e50 con 0x7fdadc137890 2024-03-27T08:26:16.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad9d9b640 1 -- 172.21.15.104:0/2403547562 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdac400e230 con 0x7fdadc137890 2024-03-27T08:26:16.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.019+0000 7fdad9d9b640 1 -- 172.21.15.104:0/2403547562 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdac400e4d0 con 0x7fdadc137890 2024-03-27T08:26:16.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 -- 172.21.15.104:0/2403547562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc137890 msgr2=0x7fdadc139cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 --2- 172.21.15.104:0/2403547562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc137890 0x7fdadc139cb0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 -- 172.21.15.104:0/2403547562 shutdown_connections 2024-03-27T08:26:16.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 --2- 172.21.15.104:0/2403547562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc137890 0x7fdadc139cb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 -- 172.21.15.104:0/2403547562 >> 172.21.15.104:0/2403547562 conn(0x7fdadc0a4ae0 msgr2=0x7fdadc0a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 -- 172.21.15.104:0/2403547562 shutdown_connections 2024-03-27T08:26:16.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 1 -- 172.21.15.104:0/2403547562 wait complete. 2024-03-27T08:26:16.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.021+0000 7fdae0fcf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.018 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.024+0000 7fdae0fcf640 1 Processor -- start 2024-03-27T08:26:16.018 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.024+0000 7fdae0fcf640 1 -- start start 2024-03-27T08:26:16.018 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.025+0000 7fdae0fcf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 0x7fdadc11e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.019 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.025+0000 7fdae0fcf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc11dae0 con 0x7fdadc131830 2024-03-27T08:26:16.019 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdada59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 0x7fdadc11e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.019 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdada59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 0x7fdadc11e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52530/0 (socket says 172.21.15.104:52530) 2024-03-27T08:26:16.019 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdada59c640 1 -- 172.21.15.104:0/993135327 learned_addr learned my addr 172.21.15.104:0/993135327 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.019 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdada59c640 1 -- 172.21.15.104:0/993135327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc11fdd0 con 0x7fdadc131830 2024-03-27T08:26:16.019 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdada59c640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 0x7fdadc11e030 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdacc004b80 con 0x7fdadc131830 2024-03-27T08:26:16.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdacc004ce0 con 0x7fdadc131830 2024-03-27T08:26:16.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.026+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdacc007120 con 0x7fdadc131830 2024-03-27T08:26:16.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.027+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc11d740 con 0x7fdadc131830 2024-03-27T08:26:16.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.027+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc11f130 con 0x7fdadc131830 2024-03-27T08:26:16.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.028+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc11d930 con 0x7fdadc131830 2024-03-27T08:26:16.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.028+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fdacc0072d0 con 0x7fdadc131830 2024-03-27T08:26:16.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.028+0000 7fdad2ffd640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdabc0387a0 0x7fdabc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.028+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fdacc048060 con 0x7fdadc131830 2024-03-27T08:26:16.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.029+0000 7fdad3fff640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdabc0387a0 0x7fdabc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.029+0000 7fdad3fff640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdabc0387a0 0x7fdabc03acb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.031 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.039+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fdacc00a9e0 con 0x7fdadc131830 2024-03-27T08:26:16.047 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.049+0000 7f23540ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.048 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.056+0000 7f23540ad640 1 Processor -- start 2024-03-27T08:26:16.048 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.056+0000 7f23540ad640 1 -- start start 2024-03-27T08:26:16.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.051+0000 7f1915b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.057+0000 7f23540ad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f234c131830 0x7f234c131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.057+0000 7f23540ad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c0735e0 con 0x7f234c134580 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f23540ad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f234c0a0260 con 0x7f234c131830 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351e49640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f234c131830 0x7f234c131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351e49640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f234c131830 0x7f234c131c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52542/0 (socket says 172.21.15.104:52542) 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351648640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f234c134580 0x7f234c134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40956/0 (socket says 172.21.15.104:40956) 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351e49640 1 -- 172.21.15.104:0/3139355782 learned_addr learned my addr 172.21.15.104:0/3139355782 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351e49640 1 -- 172.21.15.104:0/3139355782 >> v1:172.21.15.104:6789/0 conn(0x7f234c134580 legacy=0x7f234c134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351e49640 1 -- 172.21.15.104:0/3139355782 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c06eb80 con 0x7f234c131830 2024-03-27T08:26:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2351e49640 1 --2- 172.21.15.104:0/3139355782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c131c40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a426868929fdb42b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.058+0000 7f2350e47640 1 -- 172.21.15.104:0/3139355782 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2348009e50 con 0x7f234c131830 2024-03-27T08:26:16.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.059+0000 7f2350e47640 1 -- 172.21.15.104:0/3139355782 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f234800e230 con 0x7f234c131830 2024-03-27T08:26:16.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.059+0000 7f1915b5d640 1 Processor -- start 2024-03-27T08:26:16.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.059+0000 7f2350e47640 1 -- 172.21.15.104:0/3139355782 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f234800e4d0 con 0x7f234c131830 2024-03-27T08:26:16.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.059+0000 7f1915b5d640 1 -- start start 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.059+0000 7f23540ad640 1 -- 172.21.15.104:0/3139355782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 msgr2=0x7f234c131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.059+0000 7f23540ad640 1 --2- 172.21.15.104:0/3139355782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c131c40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f23540ad640 1 -- 172.21.15.104:0/3139355782 shutdown_connections 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f23540ad640 1 --2- 172.21.15.104:0/3139355782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c131c40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f23540ad640 1 -- 172.21.15.104:0/3139355782 >> 172.21.15.104:0/3139355782 conn(0x7f234c0a4ae0 msgr2=0x7f234c0a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f23540ad640 1 -- 172.21.15.104:0/3139355782 shutdown_connections 2024-03-27T08:26:16.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f23540ad640 1 -- 172.21.15.104:0/3139355782 wait complete. 2024-03-27T08:26:16.054 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f1915b5d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1910131830 0x7f1910131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.054 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f1915b5d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19100735e0 con 0x7f1910134580 2024-03-27T08:26:16.054 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f1915b5d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f19100a0260 con 0x7f1910131830 2024-03-27T08:26:16.054 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f23540ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190f7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1910131830 0x7f1910131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190f7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f1910131830 0x7f1910131c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52546/0 (socket says 172.21.15.104:52546) 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190f7fe640 1 -- 172.21.15.104:0/1215711249 learned_addr learned my addr 172.21.15.104:0/1215711249 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190effd640 1 --1- 172.21.15.104:0/1215711249 >> v1:172.21.15.104:6789/0 conn(0x7f1910134580 0x7f1910134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40970/0 (socket says 172.21.15.104:40970) 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190f7fe640 1 -- 172.21.15.104:0/1215711249 >> v1:172.21.15.104:6789/0 conn(0x7f1910134580 legacy=0x7f1910134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190e7fc640 1 -- 172.21.15.104:0/1215711249 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532885771 0 0) 0x7f19100735e0 con 0x7f1910134580 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.060+0000 7f190f7fe640 1 -- 172.21.15.104:0/1215711249 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191006eb80 con 0x7f1910131830 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.061+0000 7f190f7fe640 1 --2- 172.21.15.104:0/1215711249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910131c40 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c8d412be95fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.061+0000 7f190e7fc640 1 -- 172.21.15.104:0/1215711249 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1900009e50 con 0x7f1910131830 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.061+0000 7f190e7fc640 1 -- 172.21.15.104:0/1215711249 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f190000e230 con 0x7f1910131830 2024-03-27T08:26:16.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.061+0000 7f190e7fc640 1 -- 172.21.15.104:0/1215711249 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f190000e4d0 con 0x7f1910131830 2024-03-27T08:26:16.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.061+0000 7f23540ad640 1 Processor -- start 2024-03-27T08:26:16.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f23540ad640 1 -- start start 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 -- 172.21.15.104:0/1215711249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 msgr2=0x7f1910131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 --2- 172.21.15.104:0/1215711249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910131c40 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 -- 172.21.15.104:0/1215711249 shutdown_connections 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 --2- 172.21.15.104:0/1215711249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910131c40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 -- 172.21.15.104:0/1215711249 >> 172.21.15.104:0/1215711249 conn(0x7f19100a4ae0 msgr2=0x7f19100a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 -- 172.21.15.104:0/1215711249 shutdown_connections 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.062+0000 7f1915b5d640 1 -- 172.21.15.104:0/1215711249 wait complete. 2024-03-27T08:26:16.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f23540ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f23540ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f234c071fd0 con 0x7f234c131830 2024-03-27T08:26:16.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f1915b5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f2351e49640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f2351e49640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52554/0 (socket says 172.21.15.104:52554) 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f2351e49640 1 -- 172.21.15.104:0/2406318150 learned_addr learned my addr 172.21.15.104:0/2406318150 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.063+0000 7f2351e49640 1 -- 172.21.15.104:0/2406318150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c12f270 con 0x7f234c131830 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.064+0000 7f2351e49640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c070b80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.064+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f234800edf0 con 0x7f234c131830 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.064+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2348015420 con 0x7f234c131830 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.064+0000 7f23540ad640 1 -- 172.21.15.104:0/2406318150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c11d650 con 0x7f234c131830 2024-03-27T08:26:16.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.064+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f23480156c0 con 0x7f234c131830 2024-03-27T08:26:16.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.064+0000 7f23540ad640 1 -- 172.21.15.104:0/2406318150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c11eeb0 con 0x7f234c131830 2024-03-27T08:26:16.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.065+0000 7f1915b5d640 1 Processor -- start 2024-03-27T08:26:16.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.065+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23280050f0 con 0x7f234c131830 2024-03-27T08:26:16.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.065+0000 7f1915b5d640 1 -- start start 2024-03-27T08:26:16.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.065+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2348015820 con 0x7f234c131830 2024-03-27T08:26:16.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.065+0000 7f23427fc640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f23300387a0 0x7f233003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.066+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f23480495c0 con 0x7f234c131830 2024-03-27T08:26:16.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.066+0000 7f2351648640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f23300387a0 0x7f233003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.066+0000 7f2351648640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f23300387a0 0x7f233003acb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.063 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.067+0000 7f1915b5d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.063 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.067+0000 7f1915b5d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f191012f350 con 0x7f1910131830 2024-03-27T08:26:16.063 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.067+0000 7f190f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.063 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.067+0000 7f190f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52558/0 (socket says 172.21.15.104:52558) 2024-03-27T08:26:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.067+0000 7f190f7fe640 1 -- 172.21.15.104:0/517899923 learned_addr learned my addr 172.21.15.104:0/517899923 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.068+0000 7f190f7fe640 1 -- 172.21.15.104:0/517899923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19100727f0 con 0x7f1910131830 2024-03-27T08:26:16.064 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.068+0000 7f190f7fe640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910070cb0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.076+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f2348012a20 con 0x7f234c131830 2024-03-27T08:26:16.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.076+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f190000ed90 con 0x7f1910131830 2024-03-27T08:26:16.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.076+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1900015420 con 0x7f1910131830 2024-03-27T08:26:16.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.076+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f19000156c0 con 0x7f1910131830 2024-03-27T08:26:16.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.076+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191012f040 con 0x7f1910131830 2024-03-27T08:26:16.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.077+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191011da00 con 0x7f1910131830 2024-03-27T08:26:16.070 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.078+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f1900015820 con 0x7f1910131830 2024-03-27T08:26:16.070 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.078+0000 7f18f7fff640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f18fc0387f0 0x7f18fc03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.071 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.078+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f1900049740 con 0x7f1910131830 2024-03-27T08:26:16.071 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.078+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1910063450 con 0x7f1910131830 2024-03-27T08:26:16.071 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.078+0000 7f190effd640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f18fc0387f0 0x7f18fc03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.071 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.079+0000 7f190effd640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f18fc0387f0 0x7f18fc03ad00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.081 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.089+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f1900010c30 con 0x7f1910131830 2024-03-27T08:26:16.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.089+0000 7f0181060640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.095+0000 7f0181060640 1 Processor -- start 2024-03-27T08:26:16.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.096+0000 7f0181060640 1 -- start start 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f0181060640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f017c131650 0x7f017c131a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f0181060640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f017c0a06a0 con 0x7f017c1345b0 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f0181060640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f017c0a03e0 con 0x7f017c131650 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f017ad9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f017c131650 0x7f017c131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f0173fff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f017c1345b0 0x7f017c1349c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40986/0 (socket says 172.21.15.104:40986) 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f0173fff640 1 -- 172.21.15.104:0/3164555289 learned_addr learned my addr 172.21.15.104:0/3164555289 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f017a59c640 1 -- 172.21.15.104:0/3164555289 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889453092 0 0) 0x7f017c0a06a0 con 0x7f017c1345b0 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f017ad9d640 1 -- 172.21.15.104:0/3164555289 >> v1:172.21.15.104:6789/0 conn(0x7f017c1345b0 legacy=0x7f017c1349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.097+0000 7f017ad9d640 1 -- 172.21.15.104:0/3164555289 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c06de30 con 0x7f017c131650 2024-03-27T08:26:16.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.098+0000 7f017ad9d640 1 --2- 172.21.15.104:0/3164555289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c131a60 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fdb5bff9592954 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.098+0000 7f017a59c640 1 -- 172.21.15.104:0/3164555289 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f016c009d90 con 0x7f017c131650 2024-03-27T08:26:16.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.098+0000 7f017a59c640 1 -- 172.21.15.104:0/3164555289 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f016c00e230 con 0x7f017c131650 2024-03-27T08:26:16.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.098+0000 7f017a59c640 1 -- 172.21.15.104:0/3164555289 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f016c00e4d0 con 0x7f017c131650 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 -- 172.21.15.104:0/3164555289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 msgr2=0x7f017c131a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 --2- 172.21.15.104:0/3164555289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c131a60 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 -- 172.21.15.104:0/3164555289 shutdown_connections 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 --2- 172.21.15.104:0/3164555289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c131a60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 -- 172.21.15.104:0/3164555289 >> 172.21.15.104:0/3164555289 conn(0x7f017c0a15e0 msgr2=0x7f017c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 -- 172.21.15.104:0/3164555289 shutdown_connections 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 1 -- 172.21.15.104:0/3164555289 wait complete. 2024-03-27T08:26:16.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.099+0000 7f0181060640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:16.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.100+0000 7f0181060640 1 Processor -- start 2024-03-27T08:26:16.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.100+0000 7f0181060640 1 -- start start 2024-03-27T08:26:16.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.102+0000 7f0181060640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c120210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.102+0000 7f0181060640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1233a0 con 0x7f017c131650 2024-03-27T08:26:16.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.102+0000 7f017ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c120210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.102+0000 7f017ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c120210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52578/0 (socket says 172.21.15.104:52578) 2024-03-27T08:26:16.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.102+0000 7f017ad9d640 1 -- 172.21.15.104:0/3783856227 learned_addr learned my addr 172.21.15.104:0/3783856227 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:16.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.102+0000 7f017ad9d640 1 -- 172.21.15.104:0/3783856227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c123620 con 0x7f017c131650 2024-03-27T08:26:16.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.103+0000 7f017ad9d640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c120210 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.103+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f016c013610 con 0x7f017c131650 2024-03-27T08:26:16.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.103+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c121de0 con 0x7f017c131650 2024-03-27T08:26:16.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.103+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f016c013770 con 0x7f017c131650 2024-03-27T08:26:16.096 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.103+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f016c013a10 con 0x7f017c131650 2024-03-27T08:26:16.096 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.103+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c122d40 con 0x7f017c131650 2024-03-27T08:26:16.096 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.104+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c063450 con 0x7f017c131650 2024-03-27T08:26:16.096 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.104+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f016c013c10 con 0x7f017c131650 2024-03-27T08:26:16.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.104+0000 7f01737fe640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f015c038750 0x7f015c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:16.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.104+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f016c018040 con 0x7f017c131650 2024-03-27T08:26:16.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.105+0000 7f0173fff640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f015c038750 0x7f015c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:16.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.105+0000 7f0173fff640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f015c038750 0x7f015c03ac60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:16.102 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.110+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f016c018c30 con 0x7f017c131650 2024-03-27T08:26:16.234 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.240+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdadc063450 con 0x7fdadc131830 2024-03-27T08:26:16.234 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.241+0000 7fdad2ffd640 1 -- 172.21.15.104:0/993135327 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7fdacc00a5d0 con 0x7fdadc131830 2024-03-27T08:26:16.234 INFO:teuthology.orchestra.run.smithi104.stdout:38654705669 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdabc0387a0 msgr2=0x7fdabc03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdabc0387a0 0x7fdabc03acb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 msgr2=0x7fdadc11e030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 0x7fdadc11e030 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 shutdown_connections 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdabc0387a0 0x7fdabc03acb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 --2- 172.21.15.104:0/993135327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdadc131830 0x7fdadc11e030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.237 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 >> 172.21.15.104:0/993135327 conn(0x7fdadc0a4ae0 msgr2=0x7fdadc12fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 shutdown_connections 2024-03-27T08:26:16.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.244+0000 7fdae0fcf640 1 -- 172.21.15.104:0/993135327 wait complete. 2024-03-27T08:26:16.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-03-27T08:26:16.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.294+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f23280053a0 con 0x7f234c131830 2024-03-27T08:26:16.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.294+0000 7f23427fc640 1 -- 172.21.15.104:0/2406318150 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f2348012840 con 0x7f234c131830 2024-03-27T08:26:16.287 INFO:teuthology.orchestra.run.smithi104.stdout:38654705669 2024-03-27T08:26:16.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.296+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f017c0a2f30 con 0x7f017c131650 2024-03-27T08:26:16.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.296+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f23300387a0 msgr2=0x7f233003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.296+0000 7f232ffff640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f23300387a0 0x7f233003acb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.296+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 msgr2=0x7f234c070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.296+0000 7f232ffff640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c070b80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.297+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 shutdown_connections 2024-03-27T08:26:16.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.297+0000 7f232ffff640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f23300387a0 0x7f233003acb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.297+0000 7f232ffff640 1 --2- 172.21.15.104:0/2406318150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f234c131830 0x7f234c070b80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.297+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 >> 172.21.15.104:0/2406318150 conn(0x7f234c0a4ae0 msgr2=0x7f234c1bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.297+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 shutdown_connections 2024-03-27T08:26:16.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.297+0000 7f232ffff640 1 -- 172.21.15.104:0/2406318150 wait complete. 2024-03-27T08:26:16.290 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.298+0000 7f01737fe640 1 -- 172.21.15.104:0/3783856227 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f016c007e30 con 0x7f017c131650 2024-03-27T08:26:16.291 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:16.292 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f015c038750 msgr2=0x7f015c03ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.292 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f015c038750 0x7f015c03ac60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 msgr2=0x7f017c120210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c120210 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 shutdown_connections 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f015c038750 0x7f015c03ac60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 --2- 172.21.15.104:0/3783856227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f017c131650 0x7f017c120210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 >> 172.21.15.104:0/3783856227 conn(0x7f017c0a15e0 msgr2=0x7f017c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 shutdown_connections 2024-03-27T08:26:16.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.300+0000 7f0181060640 1 -- 172.21.15.104:0/3783856227 wait complete. 2024-03-27T08:26:16.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.305+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f191011d2e0 con 0x7f1910131830 2024-03-27T08:26:16.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.306+0000 7f18f7fff640 1 -- 172.21.15.104:0/517899923 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f1900010e60 con 0x7f1910131830 2024-03-27T08:26:16.298 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:16.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-03-27T08:26:16.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f18fc0387f0 msgr2=0x7f18fc03ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f18fc0387f0 0x7f18fc03ad00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 msgr2=0x7f1910070cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:16.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910070cb0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 shutdown_connections 2024-03-27T08:26:16.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f18fc0387f0 0x7f18fc03ad00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 --2- 172.21.15.104:0/517899923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1910131830 0x7f1910070cb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:16.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.309+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 >> 172.21.15.104:0/517899923 conn(0x7f19100a4ae0 msgr2=0x7f19100a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:16.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.310+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 shutdown_connections 2024-03-27T08:26:16.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:16.310+0000 7f1915b5d640 1 -- 172.21.15.104:0/517899923 wait complete. 2024-03-27T08:26:16.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2024-03-27T08:26:16.305 DEBUG:teuthology.parallel:result is None 2024-03-27T08:26:16.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.3 2024-03-27T08:26:16.313 DEBUG:teuthology.parallel:result is None 2024-03-27T08:26:17.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-27T08:26:17.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-27T08:26:17.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.371+0000 7f0f1c79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:17.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.377+0000 7f0f1c79d640 1 Processor -- start 2024-03-27T08:26:17.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.377+0000 7f0f1c79d640 1 -- start start 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f1c79d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0f14137890 0x7f0f14139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f1c79d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f140735e0 con 0x7f0f14131830 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f1c79d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f0f140a0260 con 0x7f0f14137890 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f19d38640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0f14137890 0x7f0f14139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f1a539640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0f14131830 0x7f0f14131c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:40996/0 (socket says 172.21.15.104:40996) 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f1a539640 1 -- 172.21.15.104:0/3669258241 learned_addr learned my addr 172.21.15.104:0/3669258241 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:17.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f19d38640 1 -- 172.21.15.104:0/3669258241 >> v1:172.21.15.104:6789/0 conn(0x7f0f14131830 legacy=0x7f0f14131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.372 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.378+0000 7f0f19d38640 1 -- 172.21.15.104:0/3669258241 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1406eb80 con 0x7f0f14137890 2024-03-27T08:26:17.372 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.379+0000 7f0f19d38640 1 --2- 172.21.15.104:0/3669258241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14137890 0x7f0f14139cb0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174afa6119cc5c95 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.372 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.379+0000 7f0f19537640 1 -- 172.21.15.104:0/3669258241 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0f04009d90 con 0x7f0f14137890 2024-03-27T08:26:17.372 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.379+0000 7f0f19537640 1 -- 172.21.15.104:0/3669258241 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0f0400e230 con 0x7f0f14137890 2024-03-27T08:26:17.372 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.379+0000 7f0f19537640 1 -- 172.21.15.104:0/3669258241 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0f0400e4d0 con 0x7f0f14137890 2024-03-27T08:26:17.373 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 -- 172.21.15.104:0/3669258241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14137890 msgr2=0x7f0f14139cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.373 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 --2- 172.21.15.104:0/3669258241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14137890 0x7f0f14139cb0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.373 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 -- 172.21.15.104:0/3669258241 shutdown_connections 2024-03-27T08:26:17.373 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 --2- 172.21.15.104:0/3669258241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14137890 0x7f0f14139cb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 -- 172.21.15.104:0/3669258241 >> 172.21.15.104:0/3669258241 conn(0x7f0f140a4ae0 msgr2=0x7f0f140a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:17.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 -- 172.21.15.104:0/3669258241 shutdown_connections 2024-03-27T08:26:17.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 1 -- 172.21.15.104:0/3669258241 wait complete. 2024-03-27T08:26:17.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.380+0000 7f0f1c79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:17.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.381+0000 7f0f1c79d640 1 Processor -- start 2024-03-27T08:26:17.374 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.381+0000 7f0f1c79d640 1 -- start start 2024-03-27T08:26:17.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.382+0000 7f0f1c79d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 0x7f0f141c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.382+0000 7f0f1c79d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14071ec0 con 0x7f0f14131830 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.382+0000 7f0f1a539640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 0x7f0f141c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.382+0000 7f0f1a539640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 0x7f0f141c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52602/0 (socket says 172.21.15.104:52602) 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.382+0000 7f0f1a539640 1 -- 172.21.15.104:0/477159420 learned_addr learned my addr 172.21.15.104:0/477159420 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f1a539640 1 -- 172.21.15.104:0/477159420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f141c91c0 con 0x7f0f14131830 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f1a539640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 0x7f0f141c9790 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0f10005a60 con 0x7f0f14131830 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0f10005bc0 con 0x7f0f14131830 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141c93f0 con 0x7f0f14131830 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0f10005e60 con 0x7f0f14131830 2024-03-27T08:26:17.376 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.383+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1411d9f0 con 0x7f0f14131830 2024-03-27T08:26:17.377 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.384+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f0f100070b0 con 0x7f0f14131830 2024-03-27T08:26:17.377 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.384+0000 7f0f0affd640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0ef80387a0 0x7f0ef803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.377 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.384+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f0f10047f40 con 0x7f0f14131830 2024-03-27T08:26:17.378 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.384+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1411d2c0 con 0x7f0f14131830 2024-03-27T08:26:17.378 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.385+0000 7f0f19d38640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0ef80387a0 0x7f0ef803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.378 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.385+0000 7f0f19d38640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0ef80387a0 0x7f0ef803acb0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.390+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f0f1000d330 con 0x7f0f14131830 2024-03-27T08:26:17.418 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.420+0000 7f75486ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:17.418 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.426+0000 7f75486ed640 1 Processor -- start 2024-03-27T08:26:17.418 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.426+0000 7f75486ed640 1 -- start start 2024-03-27T08:26:17.421 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.428+0000 7f75486ed640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7540131830 0x7f7540131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.428+0000 7f75486ed640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75400735e0 con 0x7f7540134580 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.428+0000 7f75486ed640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f75400a0260 con 0x7f7540131830 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.428+0000 7f7546489640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f7540131830 0x7f7540131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.428+0000 7f7545c88640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f7540134580 0x7f7540134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41012/0 (socket says 172.21.15.104:41012) 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.428+0000 7f7545c88640 1 -- 172.21.15.104:0/1521028660 learned_addr learned my addr 172.21.15.104:0/1521028660 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7545487640 1 -- 172.21.15.104:0/1521028660 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219879657 0 0) 0x7f75400735e0 con 0x7f7540134580 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7545487640 1 -- 172.21.15.104:0/1521028660 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f752c003580 con 0x7f7540134580 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7546489640 1 -- 172.21.15.104:0/1521028660 >> v1:172.21.15.104:6789/0 conn(0x7f7540134580 legacy=0x7f7540134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7546489640 1 -- 172.21.15.104:0/1521028660 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754006eb80 con 0x7f7540131830 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7546489640 1 --2- 172.21.15.104:0/1521028660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540131c40 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c44a97ace955a64 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7545487640 1 -- 172.21.15.104:0/1521028660 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f753800e040 con 0x7f7540131830 2024-03-27T08:26:17.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7545487640 1 -- 172.21.15.104:0/1521028660 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7538007670 con 0x7f7540131830 2024-03-27T08:26:17.423 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.429+0000 7f7545487640 1 -- 172.21.15.104:0/1521028660 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7538007910 con 0x7f7540131830 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 -- 172.21.15.104:0/1521028660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 msgr2=0x7f7540131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 --2- 172.21.15.104:0/1521028660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540131c40 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 -- 172.21.15.104:0/1521028660 shutdown_connections 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 --2- 172.21.15.104:0/1521028660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540131c40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 -- 172.21.15.104:0/1521028660 >> 172.21.15.104:0/1521028660 conn(0x7f75400a4ae0 msgr2=0x7f75400a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 -- 172.21.15.104:0/1521028660 shutdown_connections 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 1 -- 172.21.15.104:0/1521028660 wait complete. 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.430+0000 7f75486ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.431+0000 7f75486ed640 1 Processor -- start 2024-03-27T08:26:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.432+0000 7f75486ed640 1 -- start start 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.432+0000 7f75486ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.432+0000 7f75486ed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f754012f400 con 0x7f7540131830 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7546489640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7546489640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52630/0 (socket says 172.21.15.104:52630) 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7546489640 1 -- 172.21.15.104:0/945673814 learned_addr learned my addr 172.21.15.104:0/945673814 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7546489640 1 -- 172.21.15.104:0/945673814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7540072780 con 0x7f7540131830 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7546489640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540070c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.426 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f753800e040 con 0x7f7540131830 2024-03-27T08:26:17.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7540071d20 con 0x7f7540131830 2024-03-27T08:26:17.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7538013950 con 0x7f7540131830 2024-03-27T08:26:17.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7538013bf0 con 0x7f7540131830 2024-03-27T08:26:17.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.433+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754011daa0 con 0x7f7540131830 2024-03-27T08:26:17.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.434+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754011f000 con 0x7f7540131830 2024-03-27T08:26:17.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.434+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7538011040 con 0x7f7540131830 2024-03-27T08:26:17.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.435+0000 7f7536ffd640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7524038750 0x7f752403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.435+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f75380496a0 con 0x7f7540131830 2024-03-27T08:26:17.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.435+0000 7f7545c88640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7524038750 0x7f752403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.428 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.436+0000 7f7545c88640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7524038750 0x7f752403ac60 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.440+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f7538010ba0 con 0x7f7540131830 2024-03-27T08:26:17.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.574+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0f14063450 con 0x7f0f14131830 2024-03-27T08:26:17.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.575+0000 7f0f0affd640 1 -- 172.21.15.104:0/477159420 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f0f1000d150 con 0x7f0f14131830 2024-03-27T08:26:17.567 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:17.569 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0ef80387a0 msgr2=0x7f0ef803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0ef80387a0 0x7f0ef803acb0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 msgr2=0x7f0f141c9790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 0x7f0f141c9790 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 shutdown_connections 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0ef80387a0 0x7f0ef803acb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 --2- 172.21.15.104:0/477159420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f14131830 0x7f0f141c9790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 >> 172.21.15.104:0/477159420 conn(0x7f0f140a4ae0 msgr2=0x7f0f141bc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 shutdown_connections 2024-03-27T08:26:17.570 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.577+0000 7f0f1c79d640 1 -- 172.21.15.104:0/477159420 wait complete. 2024-03-27T08:26:17.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2024-03-27T08:26:17.580 DEBUG:teuthology.parallel:result is None 2024-03-27T08:26:17.615 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.622+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7540063450 con 0x7f7540131830 2024-03-27T08:26:17.615 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.623+0000 7f7536ffd640 1 -- 172.21.15.104:0/945673814 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f7538010790 con 0x7f7540131830 2024-03-27T08:26:17.615 INFO:teuthology.orchestra.run.smithi104.stdout:38654705670 2024-03-27T08:26:17.617 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7524038750 msgr2=0x7f752403ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.617 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7524038750 0x7f752403ac60 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.617 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 msgr2=0x7f7540070c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540070c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 shutdown_connections 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f7524038750 0x7f752403ac60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 --2- 172.21.15.104:0/945673814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7540131830 0x7f7540070c00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 >> 172.21.15.104:0/945673814 conn(0x7f75400a4ae0 msgr2=0x7f7540132360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 shutdown_connections 2024-03-27T08:26:17.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.625+0000 7f75486ed640 1 -- 172.21.15.104:0/945673814 wait complete. 2024-03-27T08:26:17.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-03-27T08:26:17.628 DEBUG:teuthology.parallel:result is None 2024-03-27T08:26:17.628 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-27T08:26:17.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:26:17.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.781+0000 7ff7604ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:17.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.786+0000 7ff7604ed640 1 Processor -- start 2024-03-27T08:26:17.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.786+0000 7ff7604ed640 1 -- start start 2024-03-27T08:26:17.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.787+0000 7ff7604ed640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff758137890 0x7ff758139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.787+0000 7ff7604ed640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7580735e0 con 0x7ff758131830 2024-03-27T08:26:17.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.787+0000 7ff7604ed640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7ff7580a0260 con 0x7ff758137890 2024-03-27T08:26:17.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.787+0000 7ff75e289640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff758131830 0x7ff758131c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41014/0 (socket says 172.21.15.104:41014) 2024-03-27T08:26:17.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.787+0000 7ff75e289640 1 -- 172.21.15.104:0/4109679542 learned_addr learned my addr 172.21.15.104:0/4109679542 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.787+0000 7ff75da88640 1 --2- 172.21.15.104:0/4109679542 >> v2:172.21.15.104:3300/0 conn(0x7ff758137890 0x7ff758139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75da88640 1 -- 172.21.15.104:0/4109679542 >> v1:172.21.15.104:6789/0 conn(0x7ff758131830 legacy=0x7ff758131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75da88640 1 -- 172.21.15.104:0/4109679542 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75806eb80 con 0x7ff758137890 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75d287640 1 -- 172.21.15.104:0/4109679542 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566813804 0 0) 0x7ff7580735e0 con 0x7ff758131830 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75da88640 1 --2- 172.21.15.104:0/4109679542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758137890 0x7ff758139cb0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5944dfa30c5a7048 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75d287640 1 -- 172.21.15.104:0/4109679542 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff74c009d90 con 0x7ff758137890 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75d287640 1 -- 172.21.15.104:0/4109679542 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff74c00e230 con 0x7ff758137890 2024-03-27T08:26:17.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.788+0000 7ff75d287640 1 -- 172.21.15.104:0/4109679542 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff74c00e4d0 con 0x7ff758137890 2024-03-27T08:26:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 -- 172.21.15.104:0/4109679542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758137890 msgr2=0x7ff758139cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 --2- 172.21.15.104:0/4109679542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758137890 0x7ff758139cb0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 -- 172.21.15.104:0/4109679542 shutdown_connections 2024-03-27T08:26:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 --2- 172.21.15.104:0/4109679542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758137890 0x7ff758139cb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 -- 172.21.15.104:0/4109679542 >> 172.21.15.104:0/4109679542 conn(0x7ff7580a4ae0 msgr2=0x7ff7580a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 -- 172.21.15.104:0/4109679542 shutdown_connections 2024-03-27T08:26:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 1 -- 172.21.15.104:0/4109679542 wait complete. 2024-03-27T08:26:17.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.789+0000 7ff7604ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.790+0000 7ff7604ed640 1 Processor -- start 2024-03-27T08:26:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.790+0000 7ff7604ed640 1 -- start start 2024-03-27T08:26:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.791+0000 7ff7604ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 0x7ff75811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.791+0000 7ff7604ed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff758071f80 con 0x7ff758131830 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.791+0000 7ff75e289640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 0x7ff75811d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.791+0000 7ff75e289640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 0x7ff75811d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52650/0 (socket says 172.21.15.104:52650) 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.791+0000 7ff75e289640 1 -- 172.21.15.104:0/824596629 learned_addr learned my addr 172.21.15.104:0/824596629 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff75e289640 1 -- 172.21.15.104:0/824596629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff758071c70 con 0x7ff758131830 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff75e289640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 0x7ff75811d230 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff748004cf0 con 0x7ff758131830 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff748004e50 con 0x7ff758131830 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff748007250 con 0x7ff758131830 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75811fea0 con 0x7ff758131830 2024-03-27T08:26:17.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.792+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75811ea10 con 0x7ff758131830 2024-03-27T08:26:17.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.793+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff748007400 con 0x7ff758131830 2024-03-27T08:26:17.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.793+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff758063450 con 0x7ff758131830 2024-03-27T08:26:17.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.793+0000 7ff756ffd640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff7440387a0 0x7ff74403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:17.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.793+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff748048860 con 0x7ff758131830 2024-03-27T08:26:17.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.794+0000 7ff75da88640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff7440387a0 0x7ff74403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:17.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.794+0000 7ff75da88640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff7440387a0 0x7ff74403acb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:17.791 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.798+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff7480060b0 con 0x7ff758131830 2024-03-27T08:26:17.963 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.970+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff75806e750 con 0x7ff7440387a0 2024-03-27T08:26:17.964 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.972+0000 7ff756ffd640 1 -- 172.21.15.104:0/824596629 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+29954 (crc 0 0 0) 0x7ff74c00e3a0 con 0x7ff7440387a0 2024-03-27T08:26:17.965 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:17.965 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff7440387a0 msgr2=0x7ff74403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff7440387a0 0x7ff74403acb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 msgr2=0x7ff75811d230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 0x7ff75811d230 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 shutdown_connections 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff7440387a0 0x7ff74403acb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 --2- 172.21.15.104:0/824596629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff758131830 0x7ff75811d230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 >> 172.21.15.104:0/824596629 conn(0x7ff7580a4ae0 msgr2=0x7ff75812fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 shutdown_connections 2024-03-27T08:26:17.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:17.974+0000 7ff7604ed640 1 -- 172.21.15.104:0/824596629 wait complete. 2024-03-27T08:26:17.977 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-03-27T08:26:16.208222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.921658"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.348334+0000","last_change":"2024-03-27T08:26:13.349679+0000","last_active":"2024-03-27T08:26:13.348334+0000","last_peered":"2024-03-27T08:26:13.348334+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:26:13.348334+0000","last_undegraded":"2024-03-27T08:26:13.348334+0000","last_fullsized":"2024-03-27T08:26:13.348334+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.341686+0000","last_change":"2024-03-27T08:26:13.342923+0000","last_active":"2024-03-27T08:26:13.341686+0000","last_peered":"2024-03-27T08:26:13.341686+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:26:13.341686+0000","last_undegraded":"2024-03-27T08:26:13.341686+0000","last_fullsized":"2024-03-27T08:26:13.341686+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:26:14.315289+0000","last_change":"2024-03-27T08:26:13.346210+0000","last_active":"2024-03-27T08:26:14.315289+0000","last_peered":"2024-03-27T08:26:14.315289+0000","last_clean":"2024-03-27T08:26:14.315289+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:26:14.315289+0000","last_undegraded":"2024-03-27T08:26:14.315289+0000","last_fullsized":"2024-03-27T08:26:14.315289+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.332317+0000","last_change":"2024-03-27T08:26:13.333247+0000","last_active":"2024-03-27T08:26:13.332317+0000","last_peered":"2024-03-27T08:26:13.332317+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:26:13.332317+0000","last_undegraded":"2024-03-27T08:26:13.332317+0000","last_fullsized":"2024-03-27T08:26:13.332317+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.345835+0000","last_change":"2024-03-27T08:26:13.347337+0000","last_active":"2024-03-27T08:26:13.345835+0000","last_peered":"2024-03-27T08:26:13.345835+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:26:13.345835+0000","last_undegraded":"2024-03-27T08:26:13.345835+0000","last_fullsized":"2024-03-27T08:26:13.345835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.335350+0000","last_change":"2024-03-27T08:26:13.336420+0000","last_active":"2024-03-27T08:26:13.335350+0000","last_peered":"2024-03-27T08:26:13.335350+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:26:13.335350+0000","last_undegraded":"2024-03-27T08:26:13.335350+0000","last_fullsized":"2024-03-27T08:26:13.335350+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.339847+0000","last_change":"2024-03-27T08:26:13.341429+0000","last_active":"2024-03-27T08:26:13.339847+0000","last_peered":"2024-03-27T08:26:13.339847+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:26:13.339847+0000","last_undegraded":"2024-03-27T08:26:13.339847+0000","last_fullsized":"2024-03-27T08:26:13.339847+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.345744+0000","last_change":"2024-03-27T08:26:13.347502+0000","last_active":"2024-03-27T08:26:13.345744+0000","last_peered":"2024-03-27T08:26:13.345744+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:26:13.345744+0000","last_undegraded":"2024-03-27T08:26:13.345744+0000","last_fullsized":"2024-03-27T08:26:13.345744+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:26:14.311388+0000","last_change":"2024-03-27T08:26:11.311391+0000","last_active":"2024-03-27T08:26:14.311388+0000","last_peered":"2024-03-27T08:26:14.311388+0000","last_clean":"2024-03-27T08:26:14.311388+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:26:14.311388+0000","last_undegraded":"2024-03-27T08:26:14.311388+0000","last_fullsized":"2024-03-27T08:26:14.311388+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:26:17.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:26:18.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.090+0000 7f8c71cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:18.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.095+0000 7f8c71cad640 1 Processor -- start 2024-03-27T08:26:18.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.095+0000 7f8c71cad640 1 -- start start 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c71cad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8c6c137890 0x7f8c6c139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c71cad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c6c0735e0 con 0x7f8c6c131830 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c71cad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f8c6c0a0260 con 0x7f8c6c137890 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6affd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f8c6c137890 0x7f8c6c139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6b7fe640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f8c6c131830 0x7f8c6c131c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41030/0 (socket says 172.21.15.104:41030) 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6b7fe640 1 -- 172.21.15.104:0/1389032368 learned_addr learned my addr 172.21.15.104:0/1389032368 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6affd640 1 -- 172.21.15.104:0/1389032368 >> v1:172.21.15.104:6789/0 conn(0x7f8c6c131830 legacy=0x7f8c6c131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6affd640 1 -- 172.21.15.104:0/1389032368 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c06eb80 con 0x7f8c6c137890 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6affd640 1 --2- 172.21.15.104:0/1389032368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c137890 0x7f8c6c139cb0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c4f3975599b1b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.096+0000 7f8c6a7fc640 1 -- 172.21.15.104:0/1389032368 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c5c009d90 con 0x7f8c6c137890 2024-03-27T08:26:18.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c6a7fc640 1 -- 172.21.15.104:0/1389032368 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8c5c00e230 con 0x7f8c6c137890 2024-03-27T08:26:18.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c6a7fc640 1 -- 172.21.15.104:0/1389032368 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c5c00e4d0 con 0x7f8c6c137890 2024-03-27T08:26:18.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 -- 172.21.15.104:0/1389032368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c137890 msgr2=0x7f8c6c139cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 --2- 172.21.15.104:0/1389032368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c137890 0x7f8c6c139cb0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 -- 172.21.15.104:0/1389032368 shutdown_connections 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 --2- 172.21.15.104:0/1389032368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c137890 0x7f8c6c139cb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 -- 172.21.15.104:0/1389032368 >> 172.21.15.104:0/1389032368 conn(0x7f8c6c0a4ae0 msgr2=0x7f8c6c0a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 -- 172.21.15.104:0/1389032368 shutdown_connections 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.097+0000 7f8c71cad640 1 -- 172.21.15.104:0/1389032368 wait complete. 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.098+0000 7f8c71cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:18.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.099+0000 7f8c71cad640 1 Processor -- start 2024-03-27T08:26:18.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.099+0000 7f8c71cad640 1 -- start start 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c71cad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 0x7f8c6c1c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c71cad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c071ec0 con 0x7f8c6c131830 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c6b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 0x7f8c6c1c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c6b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 0x7f8c6c1c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52658/0 (socket says 172.21.15.104:52658) 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c6b7fe640 1 -- 172.21.15.104:0/2590648350 learned_addr learned my addr 172.21.15.104:0/2590648350 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c6b7fe640 1 -- 172.21.15.104:0/2590648350 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c1c91c0 con 0x7f8c6c131830 2024-03-27T08:26:18.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.100+0000 7f8c6b7fe640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 0x7f8c6c1c9790 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:18.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.101+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c60005a60 con 0x7f8c6c131830 2024-03-27T08:26:18.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.101+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8c60005bc0 con 0x7f8c6c131830 2024-03-27T08:26:18.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.101+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6c1c93f0 con 0x7f8c6c131830 2024-03-27T08:26:18.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.101+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c60005e60 con 0x7f8c6c131830 2024-03-27T08:26:18.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.101+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6c11d9f0 con 0x7f8c6c131830 2024-03-27T08:26:18.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.102+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f8c600070b0 con 0x7f8c6c131830 2024-03-27T08:26:18.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.102+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6c11d2c0 con 0x7f8c6c131830 2024-03-27T08:26:18.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.102+0000 7f8c53fff640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8c580387a0 0x7f8c5803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:18.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.102+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f8c60047f40 con 0x7f8c6c131830 2024-03-27T08:26:18.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.102+0000 7f8c6affd640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8c580387a0 0x7f8c5803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:18.095 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.103+0000 7f8c6affd640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8c580387a0 0x7f8c5803acb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:18.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.107+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f8c6000d330 con 0x7f8c6c131830 2024-03-27T08:26:18.274 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.282+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c6c06e750 con 0x7f8c580387a0 2024-03-27T08:26:18.275 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.283+0000 7f8c53fff640 1 -- 172.21.15.104:0/2590648350 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+29954 (crc 0 0 0) 0x7f8c5c009db0 con 0x7f8c580387a0 2024-03-27T08:26:18.276 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:18.276 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:26:18.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8c580387a0 msgr2=0x7f8c5803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8c580387a0 0x7f8c5803acb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 msgr2=0x7f8c6c1c9790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 0x7f8c6c1c9790 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 shutdown_connections 2024-03-27T08:26:18.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8c580387a0 0x7f8c5803acb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 --2- 172.21.15.104:0/2590648350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c131830 0x7f8c6c1c9790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 >> 172.21.15.104:0/2590648350 conn(0x7f8c6c0a4ae0 msgr2=0x7f8c6c1bc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:18.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 shutdown_connections 2024-03-27T08:26:18.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.286+0000 7f8c71cad640 1 -- 172.21.15.104:0/2590648350 wait complete. 2024-03-27T08:26:18.289 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-03-27T08:26:18.208740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.922175"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.348334+0000","last_change":"2024-03-27T08:26:13.349679+0000","last_active":"2024-03-27T08:26:13.348334+0000","last_peered":"2024-03-27T08:26:13.348334+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:26:13.348334+0000","last_undegraded":"2024-03-27T08:26:13.348334+0000","last_fullsized":"2024-03-27T08:26:13.348334+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.341686+0000","last_change":"2024-03-27T08:26:13.342923+0000","last_active":"2024-03-27T08:26:13.341686+0000","last_peered":"2024-03-27T08:26:13.341686+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:26:13.341686+0000","last_undegraded":"2024-03-27T08:26:13.341686+0000","last_fullsized":"2024-03-27T08:26:13.341686+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:26:14.315289+0000","last_change":"2024-03-27T08:26:13.346210+0000","last_active":"2024-03-27T08:26:14.315289+0000","last_peered":"2024-03-27T08:26:14.315289+0000","last_clean":"2024-03-27T08:26:14.315289+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:26:14.315289+0000","last_undegraded":"2024-03-27T08:26:14.315289+0000","last_fullsized":"2024-03-27T08:26:14.315289+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.332317+0000","last_change":"2024-03-27T08:26:13.333247+0000","last_active":"2024-03-27T08:26:13.332317+0000","last_peered":"2024-03-27T08:26:13.332317+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:26:13.332317+0000","last_undegraded":"2024-03-27T08:26:13.332317+0000","last_fullsized":"2024-03-27T08:26:13.332317+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.345835+0000","last_change":"2024-03-27T08:26:13.347337+0000","last_active":"2024-03-27T08:26:13.345835+0000","last_peered":"2024-03-27T08:26:13.345835+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:26:13.345835+0000","last_undegraded":"2024-03-27T08:26:13.345835+0000","last_fullsized":"2024-03-27T08:26:13.345835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.335350+0000","last_change":"2024-03-27T08:26:13.336420+0000","last_active":"2024-03-27T08:26:13.335350+0000","last_peered":"2024-03-27T08:26:13.335350+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:26:13.335350+0000","last_undegraded":"2024-03-27T08:26:13.335350+0000","last_fullsized":"2024-03-27T08:26:13.335350+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.339847+0000","last_change":"2024-03-27T08:26:13.341429+0000","last_active":"2024-03-27T08:26:13.339847+0000","last_peered":"2024-03-27T08:26:13.339847+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:26:13.339847+0000","last_undegraded":"2024-03-27T08:26:13.339847+0000","last_fullsized":"2024-03-27T08:26:13.339847+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:26:13.345744+0000","last_change":"2024-03-27T08:26:13.347502+0000","last_active":"2024-03-27T08:26:13.345744+0000","last_peered":"2024-03-27T08:26:13.345744+0000","last_clean":"2024-03-27T08:26:12.271928+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:26:13.345744+0000","last_undegraded":"2024-03-27T08:26:13.345744+0000","last_fullsized":"2024-03-27T08:26:13.345744+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:26:14.311388+0000","last_change":"2024-03-27T08:26:11.311391+0000","last_active":"2024-03-27T08:26:14.311388+0000","last_peered":"2024-03-27T08:26:14.311388+0000","last_clean":"2024-03-27T08:26:14.311388+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:26:14.311388+0000","last_undegraded":"2024-03-27T08:26:14.311388+0000","last_fullsized":"2024-03-27T08:26:14.311388+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:26:18.290 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-27T08:26:18.290 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-27T08:26:18.290 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-27T08:26:18.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-27T08:26:18.398 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.403+0000 7f3405e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:18.399 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.407+0000 7f3405e6d640 1 Processor -- start 2024-03-27T08:26:18.399 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.407+0000 7f3405e6d640 1 -- start start 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f3405e6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f340012f5c0 0x7f340012f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f3405e6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34000732c0 con 0x7f3400132520 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f3405e6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f340009eeb0 con 0x7f340012f5c0 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33ff7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f340012f5c0 0x7f340012f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33f7fff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f3400132520 0x7f3400132930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:41036/0 (socket says 172.21.15.104:41036) 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33ff7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f340012f5c0 0x7f340012f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52670/0 (socket says 172.21.15.104:52670) 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33f7fff640 1 -- 172.21.15.104:0/2587039298 learned_addr learned my addr 172.21.15.104:0/2587039298 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:26:18.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33ff7fe640 1 -- 172.21.15.104:0/2587039298 >> v1:172.21.15.104:6789/0 conn(0x7f3400132520 legacy=0x7f3400132930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.402 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33ff7fe640 1 -- 172.21.15.104:0/2587039298 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34000a4820 con 0x7f340012f5c0 2024-03-27T08:26:18.402 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.408+0000 7f33feffd640 1 -- 172.21.15.104:0/2587039298 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844407766 0 0) 0x7f34000732c0 con 0x7f3400132520 2024-03-27T08:26:18.402 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f33ff7fe640 1 --2- 172.21.15.104:0/2587039298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340012f9d0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a9943a13e048ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:18.402 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f33feffd640 1 -- 172.21.15.104:0/2587039298 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f33f0009d90 con 0x7f340012f5c0 2024-03-27T08:26:18.402 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f33feffd640 1 -- 172.21.15.104:0/2587039298 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f33f000e230 con 0x7f340012f5c0 2024-03-27T08:26:18.402 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f33feffd640 1 -- 172.21.15.104:0/2587039298 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f33f000e4d0 con 0x7f340012f5c0 2024-03-27T08:26:18.403 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f3405e6d640 1 -- 172.21.15.104:0/2587039298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 msgr2=0x7f340012f9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.403 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f3405e6d640 1 --2- 172.21.15.104:0/2587039298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340012f9d0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.403 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.409+0000 7f3405e6d640 1 -- 172.21.15.104:0/2587039298 shutdown_connections 2024-03-27T08:26:18.403 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.410+0000 7f3405e6d640 1 --2- 172.21.15.104:0/2587039298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340012f9d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.403 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.410+0000 7f3405e6d640 1 -- 172.21.15.104:0/2587039298 >> 172.21.15.104:0/2587039298 conn(0x7f34000a29e0 msgr2=0x7f34000a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:18.403 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.410+0000 7f3405e6d640 1 -- 172.21.15.104:0/2587039298 shutdown_connections 2024-03-27T08:26:18.404 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.410+0000 7f3405e6d640 1 -- 172.21.15.104:0/2587039298 wait complete. 2024-03-27T08:26:18.404 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.410+0000 7f3405e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:26:18.404 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.411+0000 7f3405e6d640 1 Processor -- start 2024-03-27T08:26:18.404 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.411+0000 7f3405e6d640 1 -- start start 2024-03-27T08:26:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.412+0000 7f3405e6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340011b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.412+0000 7f3405e6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f340011e1e0 con 0x7f340012f5c0 2024-03-27T08:26:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.412+0000 7f33ff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340011b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.412+0000 7f33ff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340011b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52676/0 (socket says 172.21.15.104:52676) 2024-03-27T08:26:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.412+0000 7f33ff7fe640 1 -- 172.21.15.104:0/2639770040 learned_addr learned my addr 172.21.15.104:0/2639770040 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:26:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f33ff7fe640 1 -- 172.21.15.104:0/2639770040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3400074f00 con 0x7f340012f5c0 2024-03-27T08:26:18.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f33ff7fe640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340011b130 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:18.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f33f0014620 con 0x7f340012f5c0 2024-03-27T08:26:18.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f33f0014780 con 0x7f340012f5c0 2024-03-27T08:26:18.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f33f0014a20 con 0x7f340012f5c0 2024-03-27T08:26:18.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340011def0 con 0x7f340012f5c0 2024-03-27T08:26:18.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.413+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340011d1a0 con 0x7f340012f5c0 2024-03-27T08:26:18.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.414+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f33f0014bd0 con 0x7f340012f5c0 2024-03-27T08:26:18.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.414+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3400063450 con 0x7f340012f5c0 2024-03-27T08:26:18.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.414+0000 7f33fcff9640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f33e00387a0 0x7f33e003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:26:18.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.414+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f33f0019040 con 0x7f340012f5c0 2024-03-27T08:26:18.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.414+0000 7f33f7fff640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f33e00387a0 0x7f33e003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:26:18.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.415+0000 7f33f7fff640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f33e00387a0 0x7f33e003acb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:26:18.411 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.419+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f33f0018190 con 0x7f340012f5c0 2024-03-27T08:26:18.623 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.631+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3400074f00 con 0x7f340012f5c0 2024-03-27T08:26:18.623 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.631+0000 7f33fcff9640 1 -- 172.21.15.104:0/2639770040 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (crc 0 0 0) 0x7f3400074f00 con 0x7f340012f5c0 2024-03-27T08:26:18.624 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:26:18.624 INFO:teuthology.orchestra.run.smithi104.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f33e00387a0 msgr2=0x7f33e003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f33e00387a0 0x7f33e003acb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 msgr2=0x7f340011b130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340011b130 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 shutdown_connections 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f33e00387a0 0x7f33e003acb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 --2- 172.21.15.104:0/2639770040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f340012f5c0 0x7f340011b130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 >> 172.21.15.104:0/2639770040 conn(0x7f34000a29e0 msgr2=0x7f34001b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 shutdown_connections 2024-03-27T08:26:18.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:26:18.633+0000 7f3405e6d640 1 -- 172.21.15.104:0/2639770040 wait complete. 2024-03-27T08:26:18.636 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-27T08:26:18.636 INFO:teuthology.run_tasks:Running task workunit... 2024-03-27T08:26:18.652 INFO:tasks.workunit:Pulling workunits from ref eae914f097640d21427554e7f6aa6f1e9c4a7748 2024-03-27T08:26:18.652 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-27T08:26:18.652 DEBUG:teuthology.orchestra.run.smithi104:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-27T08:26:18.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:26:18.667 INFO:teuthology.orchestra.run.smithi104.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-27T08:26:18.667 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-27T08:26:18.722 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-27T08:26:18.722 DEBUG:teuthology.orchestra.run.smithi104:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-27T08:26:18.776 INFO:tasks.workunit:timeout=3h 2024-03-27T08:26:18.777 INFO:tasks.workunit:cleanup=True 2024-03-27T08:26:18.777 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout eae914f097640d21427554e7f6aa6f1e9c4a7748 2024-03-27T08:26:18.832 INFO:tasks.workunit.client.0.smithi104.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr:Note: switching to 'eae914f097640d21427554e7f6aa6f1e9c4a7748'. 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr:state without impacting any branches by switching back to a branch. 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.918 INFO:tasks.workunit.client.0.smithi104.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: git switch -c 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr:Or undo this operation with: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: git switch - 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr: 2024-03-27T08:27:00.919 INFO:tasks.workunit.client.0.smithi104.stderr:HEAD is now at eae914f0976 Merge pull request #56449 from afreen23/wip-security 2024-03-27T08:27:00.920 DEBUG:teuthology.orchestra.run.smithi104:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-27T08:27:00.980 INFO:tasks.workunit.client.0.smithi104.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-27T08:27:00.982 INFO:tasks.workunit.client.0.smithi104.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T08:27:00.982 INFO:tasks.workunit.client.0.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-27T08:27:01.716 INFO:tasks.workunit.client.0.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-27T08:27:01.781 INFO:tasks.workunit.client.0.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-27T08:27:01.825 INFO:tasks.workunit.client.0.smithi104.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T08:27:01.826 INFO:tasks.workunit.client.0.smithi104.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T08:27:01.826 INFO:tasks.workunit.client.0.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-27T08:27:01.866 INFO:tasks.workunit.client.0.smithi104.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T08:27:01.869 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-03-27T08:27:01.869 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-27T08:27:01.940 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-27T08:27:01.941 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-27T08:27:01.941 DEBUG:teuthology.orchestra.run.smithi104:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=eae914f097640d21427554e7f6aa6f1e9c4a7748 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-03-27T08:27:01.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ parallel=1 2024-03-27T08:27:01.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' --crimson = --serial ']' 2024-03-27T08:27:01.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ crimson=0 2024-03-27T08:27:01.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' --crimson = --crimson ']' 2024-03-27T08:27:01.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ parallel=0 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ crimson=1 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ color= 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' -t 1 ']' 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-27T08:27:02.000 INFO:tasks.workunit.client.0.smithi104.stderr:+ declare -A pids 2024-03-27T08:27:02.001 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:27:02.001 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:27:02.001 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_aio 2024-03-27T08:27:02.016 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:27:02.016 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 42 tests from 2 test suites. 2024-03-27T08:27:02.017 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:27:02.017 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 26 tests from LibRadosAio 2024-03-27T08:27:02.017 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.TooBig 2024-03-27T08:27:04.830 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.TooBig (2813 ms) 2024-03-27T08:27:04.830 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-03-27T08:27:07.858 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.SimpleWrite (3028 ms) 2024-03-27T08:27:07.858 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-03-27T08:27:10.876 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.WaitForSafe (3018 ms) 2024-03-27T08:27:10.876 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-03-27T08:27:13.886 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTrip (3009 ms) 2024-03-27T08:27:13.886 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-03-27T08:27:16.901 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTrip2 (3016 ms) 2024-03-27T08:27:16.901 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-03-27T08:27:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTrip3 (3008 ms) 2024-03-27T08:27:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-03-27T08:27:22.933 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripAppend (3025 ms) 2024-03-27T08:27:22.933 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-03-27T08:27:25.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RemoveTest (3009 ms) 2024-03-27T08:27:25.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-27T08:27:28.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3020 ms) 2024-03-27T08:27:28.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RmXattr 2024-03-27T08:27:31.971 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RmXattr (3009 ms) 2024-03-27T08:27:31.972 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.XattrIter 2024-03-27T08:27:34.986 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.XattrIter (3015 ms) 2024-03-27T08:27:34.986 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.IsComplete 2024-03-27T08:27:38.016 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.IsComplete (3030 ms) 2024-03-27T08:27:38.016 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.IsSafe 2024-03-27T08:27:41.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.IsSafe (3012 ms) 2024-03-27T08:27:41.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-03-27T08:27:44.042 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.ReturnValue (3013 ms) 2024-03-27T08:27:44.042 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.Flush 2024-03-27T08:27:47.058 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.Flush (3015 ms) 2024-03-27T08:27:47.058 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-03-27T08:27:50.068 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.FlushAsync (3012 ms) 2024-03-27T08:27:50.068 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-27T08:27:53.079 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3011 ms) 2024-03-27T08:27:53.079 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-27T08:27:56.098 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (3019 ms) 2024-03-27T08:27:56.099 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-03-27T08:27:58.824 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.SimpleStat (2726 ms) 2024-03-27T08:27:58.824 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-03-27T08:28:01.823 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.OperateMtime (2999 ms) 2024-03-27T08:28:01.823 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-03-27T08:28:04.849 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.Operate2Mtime (3026 ms) 2024-03-27T08:28:04.849 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-03-27T08:28:07.859 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.SimpleStatNS (3009 ms) 2024-03-27T08:28:07.859 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.StatRemove 2024-03-27T08:28:10.872 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.StatRemove (3013 ms) 2024-03-27T08:28:10.872 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-03-27T08:28:13.872 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.ExecuteClass (3000 ms) 2024-03-27T08:28:13.873 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-03-27T08:28:16.887 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.MultiWrite (3015 ms) 2024-03-27T08:28:16.887 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-03-27T08:28:19.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.AioUnlock (3016 ms) 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 26 tests from LibRadosAio (77887 ms total) 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 16 tests from LibRadosAioEC 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1100: Skipped 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.907 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-03-27T08:28:19.908 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-03-27T08:28:19.908 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1132: Skipped 2024-03-27T08:28:19.908 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.908 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-03-27T08:28:19.908 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1149: Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1182: Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1215: Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1286: Skipped 2024-03-27T08:28:19.909 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1326: Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1366: Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.Flush 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1385: Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-03-27T08:28:19.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1415: Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1454: Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1498: Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1532: Skipped 2024-03-27T08:28:19.911 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1593: Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1649: Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1681: Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 42 tests from 2 test suites ran. (77887 ms total) 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 26 tests. 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 16 tests, listed below: 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-03-27T08:28:19.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-03-27T08:28:19.914 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:28:19.915 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:28:19.915 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_aio_pp 2024-03-27T08:28:19.924 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:28:19.924 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 57 tests from 4 test suites. 2024-03-27T08:28:19.924 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:28:19.924 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 32 tests from LibRadosAio 2024-03-27T08:28:19.924 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-03-27T08:28:22.831 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.TooBigPP (2906 ms) 2024-03-27T08:28:22.998 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-03-27T08:28:39.844 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.PoolQuotaPP (17013 ms) 2024-03-27T08:28:39.844 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-03-27T08:28:45.860 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.SimpleWritePP (6017 ms) 2024-03-27T08:28:45.861 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-03-27T08:28:48.887 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.WaitForSafePP (3027 ms) 2024-03-27T08:28:48.888 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-03-27T08:28:51.835 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripPP (2947 ms) 2024-03-27T08:28:51.835 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-03-27T08:28:54.855 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3019 ms) 2024-03-27T08:28:54.855 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-03-27T08:28:57.861 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3007 ms) 2024-03-27T08:28:57.862 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-27T08:29:00.873 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3011 ms) 2024-03-27T08:29:00.873 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-03-27T08:29:03.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.IsCompletePP (3002 ms) 2024-03-27T08:29:03.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-03-27T08:29:06.902 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.IsSafePP (3028 ms) 2024-03-27T08:29:06.902 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-03-27T08:29:09.921 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.ReturnValuePP (3018 ms) 2024-03-27T08:29:09.921 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.FlushPP 2024-03-27T08:29:12.932 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.FlushPP (3011 ms) 2024-03-27T08:29:12.932 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-03-27T08:29:15.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3015 ms) 2024-03-27T08:29:15.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-27T08:29:18.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3011 ms) 2024-03-27T08:29:18.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-27T08:29:21.841 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (2883 ms) 2024-03-27T08:29:21.841 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-27T08:29:24.860 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3018 ms) 2024-03-27T08:29:24.860 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-27T08:29:27.879 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3019 ms) 2024-03-27T08:29:27.879 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-27T08:29:30.893 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3015 ms) 2024-03-27T08:29:30.893 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-03-27T08:29:33.895 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.SimpleStatPP (3001 ms) 2024-03-27T08:29:33.895 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-03-27T08:29:36.920 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.OperateMtime (3025 ms) 2024-03-27T08:29:36.920 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-03-27T08:29:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.OperateMtime2 (3003 ms) 2024-03-27T08:29:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-03-27T08:29:42.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.StatRemovePP (3019 ms) 2024-03-27T08:29:42.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-03-27T08:29:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3014 ms) 2024-03-27T08:29:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.OmapPP 2024-03-27T08:29:48.975 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.OmapPP (3020 ms) 2024-03-27T08:29:48.975 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-03-27T08:29:51.844 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.MultiWritePP (2869 ms) 2024-03-27T08:29:51.844 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-03-27T08:29:54.852 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.AioUnlockPP (3008 ms) 2024-03-27T08:29:54.867 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-27T08:29:57.868 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3016 ms) 2024-03-27T08:29:57.868 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-03-27T08:30:00.887 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RacingRemovePP (3019 ms) 2024-03-27T08:30:00.887 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-27T08:30:03.893 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (3006 ms) 2024-03-27T08:30:03.893 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-27T08:30:06.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3011 ms) 2024-03-27T08:30:06.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-03-27T08:30:09.409 INFO:tasks.workunit.client.0.smithi104.stdout:setting pool EIO 2024-03-27T08:30:19.075 INFO:tasks.workunit.client.0.smithi104.stdout:waiting for 754 2024-03-27T08:30:20.075 INFO:tasks.workunit.client.0.smithi104.stdout:waiting for 754 2024-03-27T08:30:21.075 INFO:tasks.workunit.client.0.smithi104.stdout:waiting for 754 2024-03-27T08:30:22.076 INFO:tasks.workunit.client.0.smithi104.stdout:max_success 246, min_failed 247 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAio.PoolEIOFlag (15948 ms) 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAio.MultiReads 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2427: Skipped 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 32 tests from LibRadosAio (122928 ms total) 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosAioPP 2024-03-27T08:30:22.853 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-27T08:30:25.869 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (3017 ms) 2024-03-27T08:30:25.869 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-27T08:30:34.864 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (8994 ms) 2024-03-27T08:30:34.864 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-03-27T08:30:49.904 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAioPP.RmXattrPP (15041 ms) 2024-03-27T08:30:49.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-27T08:30:52.921 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3017 ms) 2024-03-27T08:30:52.921 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosAioPP (30069 ms total) 2024-03-27T08:30:52.921 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:30:52.921 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosIoPP 2024-03-27T08:30:52.921 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-03-27T08:30:55.933 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.XattrListPP (3010 ms) 2024-03-27T08:30:55.933 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosIoPP (3010 ms total) 2024-03-27T08:30:55.933 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:30:55.933 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 20 tests from LibRadosAioEC 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1384: Skipped 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1419: Skipped 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1436: Skipped 2024-03-27T08:30:55.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1467: Skipped 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1500: Skipped 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-03-27T08:30:55.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1605: Skipped 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1637: Skipped 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-03-27T08:30:55.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1705: Skipped 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1742: Skipped 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1780: Skipped 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1796: Skipped 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-27T08:30:55.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1824: Skipped 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1859: Skipped 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1904: Skipped 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-03-27T08:30:55.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1946: Skipped 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1977: Skipped 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2008: Skipped 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-27T08:30:55.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2059: Skipped 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2089: Skipped 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2124: Skipped 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:30:55.941 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 57 tests from 4 test suites ran. (156008 ms total) 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 36 tests. 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 21 tests, listed below: 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-03-27T08:30:55.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-03-27T08:30:55.943 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-03-27T08:30:55.944 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:30:55.944 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:30:55.944 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_io 2024-03-27T08:30:55.956 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:30:55.956 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 24 tests from 2 test suites. 2024-03-27T08:30:55.956 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:30:55.956 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from LibRadosIo 2024-03-27T08:30:57.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-03-27T08:30:58.281 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.SimpleWrite (343 ms) 2024-03-27T08:30:58.281 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.TooBig 2024-03-27T08:30:58.281 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-03-27T08:30:58.281 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:no timeout :/ 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:no timeout :/ 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:no timeout :/ 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:no timeout :/ 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:no timeout :/ 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.ReadTimeout (67 ms) 2024-03-27T08:30:58.349 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-03-27T08:30:58.380 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.RoundTrip (32 ms) 2024-03-27T08:30:58.380 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.Checksum 2024-03-27T08:30:58.402 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.Checksum (22 ms) 2024-03-27T08:30:58.402 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-27T08:30:58.427 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (25 ms) 2024-03-27T08:30:58.427 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-27T08:30:58.453 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (26 ms) 2024-03-27T08:30:58.453 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-03-27T08:30:58.479 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.AppendRoundTrip (27 ms) 2024-03-27T08:30:58.480 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-03-27T08:30:58.484 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.ZeroLenZero (4 ms) 2024-03-27T08:30:58.484 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.TruncTest 2024-03-27T08:30:58.515 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.TruncTest (31 ms) 2024-03-27T08:30:58.515 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-03-27T08:30:58.539 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.RemoveTest (24 ms) 2024-03-27T08:30:58.539 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-27T08:30:58.569 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (30 ms) 2024-03-27T08:30:58.569 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.RmXattr 2024-03-27T08:30:58.641 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.RmXattr (72 ms) 2024-03-27T08:30:58.641 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIo.XattrIter 2024-03-27T08:30:58.678 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIo.XattrIter (37 ms) 2024-03-27T08:30:58.945 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from LibRadosIo (740 ms total) 2024-03-27T08:30:58.945 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:30:58.945 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 10 tests from LibRadosIoEC 2024-03-27T08:30:58.945 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:30:58.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-27T08:30:58.946 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.947 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-03-27T08:30:58.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 24 tests from 2 test suites ran. (2987 ms total) 2024-03-27T08:30:58.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 14 tests. 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 10 tests, listed below: 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-03-27T08:30:58.951 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:30:58.952 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:30:58.952 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_io_pp 2024-03-27T08:30:58.966 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:30:58.966 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 37 tests from 2 test suites. 2024-03-27T08:30:58.966 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:30:58.966 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 20 tests from LibRadosIoPP 2024-03-27T08:30:58.966 INFO:tasks.workunit.client.0.smithi104.stdout:seed 34686 2024-03-27T08:31:00.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-03-27T08:31:00.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-27T08:31:00.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-27T08:31:01.352 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (399 ms) 2024-03-27T08:31:01.352 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-03-27T08:31:01.418 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.ReadOpPP (65 ms) 2024-03-27T08:31:01.418 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-27T08:31:01.444 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (26 ms) 2024-03-27T08:31:01.444 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-03-27T08:31:01.463 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.RoundTripPP (20 ms) 2024-03-27T08:31:01.463 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-27T08:31:01.483 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (20 ms) 2024-03-27T08:31:01.483 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-03-27T08:31:01.501 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.Checksum (18 ms) 2024-03-27T08:31:01.502 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-27T08:31:01.522 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (21 ms) 2024-03-27T08:31:01.522 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-27T08:31:01.549 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (27 ms) 2024-03-27T08:31:01.549 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-27T08:31:01.575 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (26 ms) 2024-03-27T08:31:01.575 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-27T08:31:01.593 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (18 ms) 2024-03-27T08:31:01.593 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-27T08:31:01.622 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (29 ms) 2024-03-27T08:31:01.622 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-03-27T08:31:01.647 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.TruncTestPP (26 ms) 2024-03-27T08:31:01.648 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-27T08:31:01.672 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (24 ms) 2024-03-27T08:31:01.672 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-27T08:31:01.701 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (30 ms) 2024-03-27T08:31:01.702 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-03-27T08:31:01.774 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.RmXattrPP (72 ms) 2024-03-27T08:31:01.775 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-03-27T08:31:01.811 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.XattrListPP (37 ms) 2024-03-27T08:31:01.811 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-03-27T08:31:01.836 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.CmpExtPP (25 ms) 2024-03-27T08:31:01.836 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-27T08:31:01.851 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (15 ms) 2024-03-27T08:31:01.851 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-27T08:31:01.877 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (26 ms) 2024-03-27T08:31:02.872 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 20 tests from LibRadosIoPP (924 ms total) 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 17 tests from LibRadosIoECPP 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-03-27T08:31:02.873 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.874 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.875 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.876 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.877 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.878 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.879 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.880 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:02.881 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 17 tests from LibRadosIoECPP (0 ms total) 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 37 tests from 2 test suites ran. (3901 ms total) 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 20 tests. 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 17 tests, listed below: 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-03-27T08:31:02.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-03-27T08:31:02.883 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-03-27T08:31:02.884 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-03-27T08:31:02.884 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-03-27T08:31:02.884 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-27T08:31:02.884 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:02.884 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:02.884 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_asio 2024-03-27T08:31:02.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.917+0000 7f6d49e6b6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:02.915 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.923+0000 7f6d49e6b6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:02.915 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.923+0000 7f6d49e6b6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:02.915 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 12 tests from 1 test suite. 2024-03-27T08:31:02.916 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:02.916 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 12 tests from AsioRados 2024-03-27T08:31:02.916 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.924+0000 7f6d49e6b6c0 1 Processor -- start 2024-03-27T08:31:02.916 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.924+0000 7f6d49e6b6c0 1 -- start start 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.925+0000 7f6d49e6b6c0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x18577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.925+0000 7f6d49e6b6c0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x16a30b0 con 0x173f730 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.925+0000 7f6d4204a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x18577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.925+0000 7f6d4204a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x18577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52974/0 (socket says 172.21.15.104:52974) 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.925+0000 7f6d4204a640 1 -- 172.21.15.104:0/2600107787 learned_addr learned my addr 172.21.15.104:0/2600107787 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.926+0000 7f6d4204a640 1 -- 172.21.15.104:0/2600107787 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x16431e0 con 0x173f730 2024-03-27T08:31:02.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.926+0000 7f6d4204a640 1 --2- 172.21.15.104:0/2600107787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x18577b0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c39abfc6bafb428 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:31:02.919 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.926+0000 7f6d40847640 1 -- 172.21.15.104:0/2600107787 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6d38033040 con 0x173f730 2024-03-27T08:31:02.919 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.926+0000 7f6d40847640 1 -- 172.21.15.104:0/2600107787 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6d38034040 con 0x173f730 2024-03-27T08:31:02.919 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.926+0000 7f6d40847640 1 -- 172.21.15.104:0/2600107787 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6d3800a040 con 0x173f730 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/2600107787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 msgr2=0x18577b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/2600107787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x18577b0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/2600107787 shutdown_connections 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/2600107787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x18577b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/2600107787 >> 172.21.15.104:0/2600107787 conn(0x1657ea0 msgr2=0x17370c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/2600107787 shutdown_connections 2024-03-27T08:31:02.920 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.927+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/2600107787 wait complete. 2024-03-27T08:31:02.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.928+0000 7f6d49e6b6c0 1 Processor -- start 2024-03-27T08:31:02.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.928+0000 7f6d49e6b6c0 1 -- start start 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.929+0000 7f6d49e6b6c0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x1842020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.929+0000 7f6d49e6b6c0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x16a0b30 con 0x173f730 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.929+0000 7f6d4204a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x1842020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.929+0000 7f6d4204a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x1842020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52990/0 (socket says 172.21.15.104:52990) 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.929+0000 7f6d4204a640 1 -- 172.21.15.104:0/3970080023 learned_addr learned my addr 172.21.15.104:0/3970080023 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d4204a640 1 -- 172.21.15.104:0/3970080023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1635e30 con 0x173f730 2024-03-27T08:31:02.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d4204a640 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x1842020 crc :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:31:02.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6d3802eb30 con 0x173f730 2024-03-27T08:31:02.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1635890 con 0x173f730 2024-03-27T08:31:02.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6d3802ec90 con 0x173f730 2024-03-27T08:31:02.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6d380300b0 con 0x173f730 2024-03-27T08:31:02.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.930+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1845240 con 0x173f730 2024-03-27T08:31:02.924 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.931+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f6d38030260 con 0x173f730 2024-03-27T08:31:02.924 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.932+0000 7f6d337fe640 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6d20036fe0 0x7f6d200394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:31:02.924 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.932+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 2672+0+0 (crc 0 0 0) 0x7f6d3806b2c0 con 0x173f730 2024-03-27T08:31:02.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.932+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x16971c0 con 0x173f730 2024-03-27T08:31:02.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.932+0000 7f6d41849640 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6d20036fe0 0x7f6d200394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:31:02.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.932+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=235) ==== 24+0+0 (crc 0 0 0) 0x7f6d3802cee0 con 0x173f730 2024-03-27T08:31:02.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.932+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x16a8a30 con 0x173f730 2024-03-27T08:31:02.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:02.933+0000 7f6d41849640 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6d20036fe0 0x7f6d200394f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:31:03.868 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.875+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v236) ==== 43+0+0 (crc 0 0 0) 0x7f6d3802c8c0 con 0x173f730 2024-03-27T08:31:03.868 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.875+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=236}) -- 0x7f6d2003b960 con 0x173f730 2024-03-27T08:31:03.868 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.876+0000 7f6d337fe640 1 -- 172.21.15.104:0/3970080023 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(236..236 src has 1..236) ==== 651+0+0 (crc 0 0 0) 0x7f6d3802caa0 con 0x173f730 2024-03-27T08:31:03.868 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.876+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> v2:172.21.15.104:6801/1224298329 conn(0x1740080 0x186c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:31:03.868 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.876+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1845990 con 0x1740080 2024-03-27T08:31:03.871 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.879+0000 7f6d4284b640 1 --2- 172.21.15.104:0/3970080023 >> v2:172.21.15.104:6801/1224298329 conn(0x1740080 0x186c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:31:03.968 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:03.976+0000 7f6d4284b640 1 --2- 172.21.15.104:0/3970080023 >> v2:172.21.15.104:6801/1224298329 conn(0x1740080 0x186c6d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:31:05.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.141+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(2 exist [writefull 0~5] v237'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.134 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-03-27T08:31:05.134 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.141+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x18552a0 con 0x1740080 2024-03-27T08:31:05.134 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.141+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1855b40 con 0x1740080 2024-03-27T08:31:05.151 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.159+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 2 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.151 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.159+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.152 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncReadCallback (18 ms) 2024-03-27T08:31:05.152 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-03-27T08:31:05.152 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.159+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1855f20 con 0x1740080 2024-03-27T08:31:05.152 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.159+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x186cc10 con 0x1740080 2024-03-27T08:31:05.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.170+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.172+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.164 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncReadFuture (13 ms) 2024-03-27T08:31:05.164 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-03-27T08:31:05.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.172+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1855670 con 0x1740080 2024-03-27T08:31:05.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.172+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x18adab0 con 0x1740080 2024-03-27T08:31:05.175 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.183+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 6 ==== osd_op_reply(7 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.177 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.185+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 7 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.177 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncReadYield (13 ms) 2024-03-27T08:31:05.177 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-03-27T08:31:05.177 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.185+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x18426b0 con 0x1740080 2024-03-27T08:31:05.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.205+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 8 ==== osd_op_reply(9 exist [write 0~5] v237'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.197 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncWriteCallback (20 ms) 2024-03-27T08:31:05.197 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-03-27T08:31:05.198 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.205+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x18426b0 con 0x1740080 2024-03-27T08:31:05.209 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.217+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 9 ==== osd_op_reply(10 exist [write 0~5] v237'3 uv3 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.210 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncWriteFuture (12 ms) 2024-03-27T08:31:05.210 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-03-27T08:31:05.210 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.217+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x18907b0 con 0x1740080 2024-03-27T08:31:05.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.227+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 10 ==== osd_op_reply(11 exist [write 0~5] v237'4 uv4 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.220 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncWriteYield (10 ms) 2024-03-27T08:31:05.220 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-27T08:31:05.220 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.227+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x186d420 con 0x1740080 2024-03-27T08:31:05.220 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.227+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x186dd80 con 0x1740080 2024-03-27T08:31:05.225 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.233+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.226 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.234+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.226 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (7 ms) 2024-03-27T08:31:05.226 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-27T08:31:05.227 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.234+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x186d8d0 con 0x1740080 2024-03-27T08:31:05.227 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.234+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1890e30 con 0x1740080 2024-03-27T08:31:05.233 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.241+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 13 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.233 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.241+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.233 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (7 ms) 2024-03-27T08:31:05.233 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-03-27T08:31:05.233 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.241+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1890140 con 0x1740080 2024-03-27T08:31:05.234 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.241+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1890e30 con 0x1740080 2024-03-27T08:31:05.239 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.247+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 15 ==== osd_op_reply(16 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.239 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.247+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 16 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.239 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncReadOperationYield (6 ms) 2024-03-27T08:31:05.239 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-27T08:31:05.240 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.247+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1890510 con 0x1740080 2024-03-27T08:31:05.249 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.257+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 17 ==== osd_op_reply(18 exist [writefull 0~5] v237'5 uv5 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.250 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (10 ms) 2024-03-27T08:31:05.250 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-27T08:31:05.250 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.257+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1890510 con 0x1740080 2024-03-27T08:31:05.259 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.267+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 18 ==== osd_op_reply(19 exist [writefull 0~5] v237'6 uv6 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.259 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (10 ms) 2024-03-27T08:31:05.260 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-27T08:31:05.260 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.267+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1890140 con 0x1740080 2024-03-27T08:31:05.269 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d4284b640 1 -- 172.21.15.104:0/3970080023 <== osd.1 v2:172.21.15.104:6801/1224298329 19 ==== osd_op_reply(20 exist [writefull 0~5] v237'7 uv7 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f6d3400a430 con 0x1740080 2024-03-27T08:31:05.269 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (10 ms) 2024-03-27T08:31:05.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 >> v2:172.21.15.104:6801/1224298329 conn(0x1740080 msgr2=0x186c6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:31:05.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> v2:172.21.15.104:6801/1224298329 conn(0x1740080 0x186c6d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:05.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6d20036fe0 msgr2=0x7f6d200394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:31:05.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6d20036fe0 0x7f6d200394f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:05.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 msgr2=0x1842020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x1842020 crc :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 shutdown_connections 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6d20036fe0 0x7f6d200394f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> v2:172.21.15.104:6801/1224298329 conn(0x1740080 0x186c6d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 --2- 172.21.15.104:0/3970080023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x173f730 0x1842020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 >> 172.21.15.104:0/3970080023 conn(0x1657ea0 msgr2=0x1739960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 shutdown_connections 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.277+0000 7f6d49e6b6c0 1 -- 172.21.15.104:0/3970080023 wait complete. 2024-03-27T08:31:05.271 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 12 tests from AsioRados (136 ms total) 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 12 tests from 1 test suite ran. (2354 ms total) 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 12 tests. 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:05.272 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_list 2024-03-27T08:31:05.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.308+0000 7f7da10449c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:05.301 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.308+0000 7f7da10449c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:05.301 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:05.308+0000 7f7da10449c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:05.301 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 11 tests from 3 test suites. 2024-03-27T08:31:05.301 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:05.301 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from LibRadosList 2024-03-27T08:31:06.887 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.ListObjects 2024-03-27T08:31:07.527 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.ListObjects (643 ms) 2024-03-27T08:31:07.527 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-27T08:31:07.735 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (208 ms) 2024-03-27T08:31:07.735 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-03-27T08:31:07.821 INFO:tasks.workunit.client.0.smithi104.stdout:myset foo1,foo2,foo3 2024-03-27T08:31:07.979 INFO:tasks.workunit.client.0.smithi104.stdout:foo1 2024-03-27T08:31:07.979 INFO:tasks.workunit.client.0.smithi104.stdout:foo2 2024-03-27T08:31:07.979 INFO:tasks.workunit.client.0.smithi104.stdout:foo3 2024-03-27T08:31:07.979 INFO:tasks.workunit.client.0.smithi104.stdout:myset foo1,foo4,foo5 2024-03-27T08:31:08.073 INFO:tasks.workunit.client.0.smithi104.stdout:foo4 2024-03-27T08:31:08.073 INFO:tasks.workunit.client.0.smithi104.stdout:foo5 2024-03-27T08:31:08.073 INFO:tasks.workunit.client.0.smithi104.stdout:foo1 2024-03-27T08:31:08.073 INFO:tasks.workunit.client.0.smithi104.stdout:myset foo6,foo7 2024-03-27T08:31:08.167 INFO:tasks.workunit.client.0.smithi104.stdout:foo7 2024-03-27T08:31:08.167 INFO:tasks.workunit.client.0.smithi104.stdout:foo6 2024-03-27T08:31:08.167 INFO:tasks.workunit.client.0.smithi104.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-27T08:31:08.261 INFO:tasks.workunit.client.0.smithi104.stdout:ns1:foo4 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout:ns1:foo5 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout:ns2:foo7 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout:ns2:foo6 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout:ns1:foo1 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout::foo1 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout::foo2 2024-03-27T08:31:08.262 INFO:tasks.workunit.client.0.smithi104.stdout::foo3 2024-03-27T08:31:08.435 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.ListObjectsNS (700 ms) 2024-03-27T08:31:08.435 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:1 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:10 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:13 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:7 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:14 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:0 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:15 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:11 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:5 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:8 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:6 0 2024-03-27T08:31:08.700 INFO:tasks.workunit.client.0.smithi104.stdout:3 0 2024-03-27T08:31:08.701 INFO:tasks.workunit.client.0.smithi104.stdout:4 0 2024-03-27T08:31:08.701 INFO:tasks.workunit.client.0.smithi104.stdout:12 0 2024-03-27T08:31:08.701 INFO:tasks.workunit.client.0.smithi104.stdout:9 0 2024-03-27T08:31:08.701 INFO:tasks.workunit.client.0.smithi104.stdout:2 0 2024-03-27T08:31:08.795 INFO:tasks.workunit.client.0.smithi104.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-27T08:31:09.046 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.ListObjectsStart (611 ms) 2024-03-27T08:31:09.046 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-03-27T08:31:09.209 INFO:tasks.workunit.client.0.smithi104.stdout:x cursor=MIN 2024-03-27T08:31:09.304 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-03-27T08:31:09.304 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-03-27T08:31:09.305 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-03-27T08:31:09.306 INFO:tasks.workunit.client.0.smithi104.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-03-27T08:31:09.513 INFO:tasks.workunit.client.0.smithi104.stdout:FIRST> seek to MIN oid=1 2024-03-27T08:31:09.647 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-03-27T08:31:09.647 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:02547ec2:::1:head 2024-03-27T08:31:09.740 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:02547ec2:::1:head -> 1 2024-03-27T08:31:09.740 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-03-27T08:31:09.740 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:52ea6a34:::10:head 2024-03-27T08:31:09.805 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:52ea6a34:::10:head -> 10 2024-03-27T08:31:09.805 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-03-27T08:31:09.805 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:566253c9:::13:head 2024-03-27T08:31:09.869 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:566253c9:::13:head -> 13 2024-03-27T08:31:09.869 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-03-27T08:31:09.869 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:5c6b0b28:::7:head 2024-03-27T08:31:09.930 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:5c6b0b28:::7:head -> 7 2024-03-27T08:31:09.930 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-03-27T08:31:09.930 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:62a1935d:::14:head 2024-03-27T08:31:09.989 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:62a1935d:::14:head -> 14 2024-03-27T08:31:09.989 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-03-27T08:31:09.989 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:6cac518f:::0:head 2024-03-27T08:31:10.045 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:6cac518f:::0:head -> 0 2024-03-27T08:31:10.045 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-03-27T08:31:10.045 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:863748b0:::15:head 2024-03-27T08:31:10.092 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:863748b0:::15:head -> 15 2024-03-27T08:31:10.092 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-03-27T08:31:10.092 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:89d3ae78:::11:head 2024-03-27T08:31:10.136 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:89d3ae78:::11:head -> 11 2024-03-27T08:31:10.136 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-03-27T08:31:10.137 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:b29083e3:::5:head 2024-03-27T08:31:10.165 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:b29083e3:::5:head -> 5 2024-03-27T08:31:10.165 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-03-27T08:31:10.165 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:bd63b0f1:::8:head 2024-03-27T08:31:10.191 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:bd63b0f1:::8:head -> 8 2024-03-27T08:31:10.192 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-03-27T08:31:10.192 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:c4fdafeb:::6:head 2024-03-27T08:31:10.215 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:c4fdafeb:::6:head -> 6 2024-03-27T08:31:10.215 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-03-27T08:31:10.215 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:cfc208b3:::3:head 2024-03-27T08:31:10.235 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:cfc208b3:::3:head -> 3 2024-03-27T08:31:10.235 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-03-27T08:31:10.235 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:d83876eb:::4:head 2024-03-27T08:31:10.249 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:d83876eb:::4:head -> 4 2024-03-27T08:31:10.249 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-03-27T08:31:10.249 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:de5d7c5f:::12:head 2024-03-27T08:31:10.264 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:de5d7c5f:::12:head -> 12 2024-03-27T08:31:10.264 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-03-27T08:31:10.264 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:e960b815:::9:head 2024-03-27T08:31:10.273 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:e960b815:::9:head -> 9 2024-03-27T08:31:10.273 INFO:tasks.workunit.client.0.smithi104.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-03-27T08:31:10.273 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:f905c69b:::2:head 2024-03-27T08:31:10.276 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:f905c69b:::2:head -> 2 2024-03-27T08:31:10.276 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:5c6b0b28:::7:head 2024-03-27T08:31:10.276 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-03-27T08:31:10.338 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:5c6b0b28:::7:head -> 7 2024-03-27T08:31:10.338 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=7 expected=7 2024-03-27T08:31:10.339 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:52ea6a34:::10:head 2024-03-27T08:31:10.339 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-03-27T08:31:10.403 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:52ea6a34:::10:head -> 10 2024-03-27T08:31:10.404 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=10 expected=10 2024-03-27T08:31:10.404 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:f905c69b:::2:head 2024-03-27T08:31:10.404 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-03-27T08:31:10.406 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:f905c69b:::2:head -> 2 2024-03-27T08:31:10.406 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=2 expected=2 2024-03-27T08:31:10.406 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:e960b815:::9:head 2024-03-27T08:31:10.406 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-03-27T08:31:10.415 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:e960b815:::9:head -> 9 2024-03-27T08:31:10.415 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=9 expected=9 2024-03-27T08:31:10.415 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:c4fdafeb:::6:head 2024-03-27T08:31:10.415 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-03-27T08:31:10.438 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:c4fdafeb:::6:head -> 6 2024-03-27T08:31:10.438 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=6 expected=6 2024-03-27T08:31:10.438 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:62a1935d:::14:head 2024-03-27T08:31:10.438 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-03-27T08:31:10.499 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:62a1935d:::14:head -> 14 2024-03-27T08:31:10.499 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=14 expected=14 2024-03-27T08:31:10.499 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:02547ec2:::1:head 2024-03-27T08:31:10.499 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-03-27T08:31:10.594 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:02547ec2:::1:head -> 1 2024-03-27T08:31:10.594 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=1 expected=1 2024-03-27T08:31:10.595 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:de5d7c5f:::12:head 2024-03-27T08:31:10.595 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-03-27T08:31:10.609 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:de5d7c5f:::12:head -> 12 2024-03-27T08:31:10.609 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=12 expected=12 2024-03-27T08:31:10.609 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:d83876eb:::4:head 2024-03-27T08:31:10.609 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-03-27T08:31:10.624 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:d83876eb:::4:head -> 4 2024-03-27T08:31:10.624 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=4 expected=4 2024-03-27T08:31:10.624 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:cfc208b3:::3:head 2024-03-27T08:31:10.624 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-03-27T08:31:10.644 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:cfc208b3:::3:head -> 3 2024-03-27T08:31:10.645 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=3 expected=3 2024-03-27T08:31:10.645 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:bd63b0f1:::8:head 2024-03-27T08:31:10.645 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-03-27T08:31:10.671 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:bd63b0f1:::8:head -> 8 2024-03-27T08:31:10.672 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=8 expected=8 2024-03-27T08:31:10.672 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:b29083e3:::5:head 2024-03-27T08:31:10.672 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-03-27T08:31:10.701 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:b29083e3:::5:head -> 5 2024-03-27T08:31:10.701 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=5 expected=5 2024-03-27T08:31:10.701 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:89d3ae78:::11:head 2024-03-27T08:31:10.701 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-03-27T08:31:10.745 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:89d3ae78:::11:head -> 11 2024-03-27T08:31:10.745 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=11 expected=11 2024-03-27T08:31:10.745 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:863748b0:::15:head 2024-03-27T08:31:10.745 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-03-27T08:31:10.793 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:863748b0:::15:head -> 15 2024-03-27T08:31:10.794 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=15 expected=15 2024-03-27T08:31:10.794 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:6cac518f:::0:head 2024-03-27T08:31:10.794 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-03-27T08:31:10.850 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:6cac518f:::0:head -> 0 2024-03-27T08:31:10.851 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=0 expected=0 2024-03-27T08:31:10.851 INFO:tasks.workunit.client.0.smithi104.stdout:: seek to 76:566253c9:::13:head 2024-03-27T08:31:10.851 INFO:tasks.workunit.client.0.smithi104.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-03-27T08:31:10.917 INFO:tasks.workunit.client.0.smithi104.stdout:> 76:566253c9:::13:head -> 13 2024-03-27T08:31:10.917 INFO:tasks.workunit.client.0.smithi104.stdout:: entry=13 expected=13 2024-03-27T08:31:11.177 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.ListObjectsCursor (2129 ms) 2024-03-27T08:31:11.177 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-03-27T08:31:11.688 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.EnumerateObjects (513 ms) 2024-03-27T08:31:11.689 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-27T08:31:11.853 INFO:tasks.workunit.client.0.smithi104.stdout:split 0/5 -> MIN 76:33333333::::head 2024-03-27T08:31:11.874 INFO:tasks.workunit.client.0.smithi104.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-03-27T08:31:11.896 INFO:tasks.workunit.client.0.smithi104.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-03-27T08:31:11.920 INFO:tasks.workunit.client.0.smithi104.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-03-27T08:31:11.941 INFO:tasks.workunit.client.0.smithi104.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-03-27T08:31:12.215 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (526 ms) 2024-03-27T08:31:12.904 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from LibRadosList (5330 ms total) 2024-03-27T08:31:12.904 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:12.904 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosListEC 2024-03-27T08:31:12.904 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:70: Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:84: Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:96: Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:84: Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:96: Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:84: Skipped 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:96: Skipped 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:78: Skipped 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosListNP 2024-03-27T08:31:12.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosListNP.ListObjectsError (3010 ms) 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosListNP (3010 ms total) 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 11 tests from 3 test suites ran. (10613 ms total) 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 8 tests. 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 3 tests, listed below: 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-03-27T08:31:15.913 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-03-27T08:31:15.914 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-03-27T08:31:15.915 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:15.915 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:15.915 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_lock 2024-03-27T08:31:15.932 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:31:15.932 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-27T08:31:15.932 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:15.932 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLock 2024-03-27T08:31:17.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-03-27T08:31:18.324 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.LockExclusive (401 ms) 2024-03-27T08:31:18.325 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.LockShared 2024-03-27T08:31:18.346 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.LockShared (22 ms) 2024-03-27T08:31:18.346 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-03-27T08:31:19.410 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.LockExclusiveDur (1064 ms) 2024-03-27T08:31:19.410 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-03-27T08:31:20.448 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.LockSharedDur (1037 ms) 2024-03-27T08:31:20.448 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-03-27T08:31:20.488 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.LockMayRenew (41 ms) 2024-03-27T08:31:20.488 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.Unlock 2024-03-27T08:31:20.534 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.Unlock (46 ms) 2024-03-27T08:31:20.534 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.ListLockers 2024-03-27T08:31:20.588 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.ListLockers (54 ms) 2024-03-27T08:31:20.588 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLock.BreakLock 2024-03-27T08:31:20.620 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLock.BreakLock (32 ms) 2024-03-27T08:31:20.934 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLock (2697 ms total) 2024-03-27T08:31:20.934 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:20.934 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLockEC 2024-03-27T08:31:20.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:31:20.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-27T08:31:20.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.937 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLockEC (0 ms total) 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:20.938 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 16 tests from 2 test suites ran. (5000 ms total) 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 8 tests. 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 8 tests, listed below: 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:20.939 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_lock_pp 2024-03-27T08:31:20.954 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:31:20.955 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-27T08:31:20.955 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:20.955 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLockPP 2024-03-27T08:31:20.955 INFO:tasks.workunit.client.0.smithi104.stdout:seed 34801 2024-03-27T08:31:22.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-27T08:31:23.319 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (381 ms) 2024-03-27T08:31:23.319 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-03-27T08:31:23.342 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.LockSharedPP (23 ms) 2024-03-27T08:31:23.342 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-27T08:31:24.380 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (1038 ms) 2024-03-27T08:31:24.380 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-27T08:31:25.503 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1119 ms) 2024-03-27T08:31:25.503 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-27T08:31:25.532 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (33 ms) 2024-03-27T08:31:25.532 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-03-27T08:31:25.570 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.UnlockPP (38 ms) 2024-03-27T08:31:25.570 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-03-27T08:31:25.614 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.ListLockersPP (44 ms) 2024-03-27T08:31:25.614 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-03-27T08:31:25.642 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosLockPP.BreakLockPP (28 ms) 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLockPP (2704 ms total) 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLockECPP 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (0 ms) 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-03-27T08:31:25.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (0 ms) 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosLockECPP (0 ms total) 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 16 tests from 2 test suites ran. (5005 ms total) 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 8 tests. 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 8 tests, listed below: 2024-03-27T08:31:25.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:25.967 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_misc 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:25.997+0000 7f38d38596c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:25.997+0000 7f38d38596c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:25.997+0000 7f38d38596c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 12 tests from 4 test suites. 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscVersion 2024-03-27T08:31:25.990 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-03-27T08:31:25.991 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-27T08:31:25.991 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-27T08:31:25.991 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:25.991 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-03-27T08:31:25.991 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-27T08:31:26.021 INFO:tasks.workunit.client.0.smithi104.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-27T08:31:26.021 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:26.029+0000 7f38d38596c0 -1 failed for service _ceph-mon._tcp 2024-03-27T08:31:26.021 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:31:26.029+0000 7f38d38596c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-27T08:31:26.021 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (31 ms) 2024-03-27T08:31:26.022 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-27T08:31:28.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2007 ms) 2024-03-27T08:31:28.029 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (2038 ms total) 2024-03-27T08:31:28.029 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:28.029 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscPool 2024-03-27T08:31:28.029 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-27T08:31:30.136 INFO:tasks.workunit.client.0.smithi104.stdout:started 0x7f389c005de0 2024-03-27T08:31:31.137 INFO:tasks.workunit.client.0.smithi104.stdout:started 0x7f38b8005de0 2024-03-27T08:31:31.137 INFO:tasks.workunit.client.0.smithi104.stdout: started 2 aios 2024-03-27T08:31:31.137 INFO:tasks.workunit.client.0.smithi104.stdout:waiting 0x7f389c005de0 2024-03-27T08:31:31.137 INFO:tasks.workunit.client.0.smithi104.stdout:waiting 0x7f38b8005de0 2024-03-27T08:31:31.140 INFO:tasks.workunit.client.0.smithi104.stdout:done. 2024-03-27T08:31:32.150 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (4122 ms) 2024-03-27T08:31:32.151 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscPool (4122 ms total) 2024-03-27T08:31:32.151 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:31:32.151 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosMisc 2024-03-27T08:31:34.160 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-03-27T08:31:34.160 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-27T08:31:34.160 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.Exec 2024-03-27T08:31:34.554 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.Exec (393 ms) 2024-03-27T08:31:34.554 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-03-27T08:31:34.594 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.WriteSame (40 ms) 2024-03-27T08:31:34.594 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-03-27T08:31:34.623 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.CmpExt (30 ms) 2024-03-27T08:31:34.623 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.Applications 2024-03-27T08:31:38.867 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.Applications (4244 ms) 2024-03-27T08:31:38.867 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-03-27T08:31:38.867 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-27T08:31:38.868 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-03-27T08:31:38.868 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-27T08:31:38.868 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-03-27T08:32:04.147 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMisc.ShutdownRace (25280 ms) 2024-03-27T08:32:04.895 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 8 tests from LibRadosMisc (29987 ms total) 2024-03-27T08:32:04.895 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:04.895 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:04.895 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 12 tests from 4 test suites ran. (38905 ms total) 2024-03-27T08:32:04.895 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 12 tests. 2024-03-27T08:32:04.908 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:04.908 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:04.908 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_misc_pp 2024-03-27T08:32:04.941 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:32:04.949+0000 7f297da79840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:04.941 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:32:04.949+0000 7f297da79840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:04.941 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:32:04.949+0000 7f297da79840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 31 tests from 7 test suites. 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscVersion 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:04.942 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 22 tests from LibRadosMiscPP 2024-03-27T08:32:04.943 INFO:tasks.workunit.client.0.smithi104.stdout:seed 96466 2024-03-27T08:32:06.900 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-27T08:32:06.900 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-03-27T08:32:06.900 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-03-27T08:32:07.388 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.LongNamePP (488 ms) 2024-03-27T08:32:07.389 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-27T08:32:07.528 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.LongLocatorPP (140 ms) 2024-03-27T08:32:07.528 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-27T08:32:07.581 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (53 ms) 2024-03-27T08:32:07.581 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-27T08:32:07.622 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (41 ms) 2024-03-27T08:32:07.622 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-03-27T08:32:07.638 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.ExecPP (16 ms) 2024-03-27T08:32:07.638 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-27T08:32:07.652 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (14 ms) 2024-03-27T08:32:07.652 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-03-27T08:32:07.679 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.Operate1PP (27 ms) 2024-03-27T08:32:07.679 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-03-27T08:32:07.694 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.Operate2PP (15 ms) 2024-03-27T08:32:07.694 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-27T08:32:07.736 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (42 ms) 2024-03-27T08:32:07.736 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-27T08:32:07.752 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (16 ms) 2024-03-27T08:32:07.752 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-27T08:32:07.774 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (22 ms) 2024-03-27T08:32:07.774 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-27T08:32:07.817 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (43 ms) 2024-03-27T08:32:07.817 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-27T08:32:07.828 INFO:tasks.workunit.client.0.smithi104.stdout:osd_max_attr_size = 0 2024-03-27T08:32:07.828 INFO:tasks.workunit.client.0.smithi104.stdout:osd_max_attr_size == 0; skipping test 2024-03-27T08:32:24.862 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (17045 ms) 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:386: Skipped 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (0 ms) 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:549: Skipped 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-03-27T08:32:24.863 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-27T08:32:24.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (48 ms) 2024-03-27T08:32:24.910 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-27T08:32:24.933 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (23 ms) 2024-03-27T08:32:24.933 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.Applications (4941 ms) 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-03-27T08:32:29.875 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-27T08:32:30.881 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 22 tests from LibRadosMiscPP (22974 ms total) 2024-03-27T08:32:30.881 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:30.881 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:476: Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:498: Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:504: Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:491: Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-03-27T08:32:30.882 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:30.883 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-27T08:32:32.890 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-03-27T08:32:33.245 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosChecksum/0.Subset (355 ms) 2024-03-27T08:32:33.245 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-03-27T08:32:33.281 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosChecksum/0.Chunked (36 ms) 2024-03-27T08:32:33.890 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosChecksum/0 (391 ms total) 2024-03-27T08:32:33.891 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:33.891 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-27T08:32:35.904 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-03-27T08:32:36.197 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosChecksum/1.Subset (293 ms) 2024-03-27T08:32:36.197 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-03-27T08:32:36.264 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosChecksum/1.Chunked (67 ms) 2024-03-27T08:32:36.873 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosChecksum/1 (360 ms total) 2024-03-27T08:32:36.873 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:36.873 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-27T08:32:38.918 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-03-27T08:32:39.289 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosChecksum/2.Subset (372 ms) 2024-03-27T08:32:39.289 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-03-27T08:32:39.306 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosChecksum/2.Chunked (17 ms) 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosChecksum/2 (389 ms total) 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscECPP 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:32:39.922 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange (0 ms) 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosMiscECPP (0 ms total) 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 31 tests from 7 test suites ran. (34979 ms total) 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 27 tests. 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP 2024-03-27T08:32:39.923 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-27T08:32:39.924 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange 2024-03-27T08:32:39.924 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:39.924 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:39.924 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_tier_pp 2024-03-27T08:32:39.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:32:39.961+0000 7fea2d6309c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:39.954 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:32:39.961+0000 7fea2d6309c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:39.954 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:32:39.961+0000 7fea2d6309c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:39.954 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 77 tests from 4 test suites. 2024-03-27T08:32:39.954 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:39.954 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosTierPP 2024-03-27T08:32:39.955 INFO:tasks.workunit.client.0.smithi104.stdout:seed 96538 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierPP.Dirty 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:287: Skipped 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:2182: Skipped 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2024-03-27T08:32:41.938 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierPP.HitSetNone 2024-03-27T08:32:41.939 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:2561: Skipped 2024-03-27T08:32:41.939 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:41.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone (0 ms) 2024-03-27T08:32:42.942 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosTierPP (0 ms total) 2024-03-27T08:32:42.942 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:42.942 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 48 tests from LibRadosTwoPoolsPP 2024-03-27T08:32:44.947 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.948 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-27T08:32:44.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (0 ms) 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush (1 ms) 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (0 ms) 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (0 ms) 2024-03-27T08:32:44.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (0 ms) 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-27T08:32:44.963 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.964 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-27T08:32:44.965 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (0 ms) 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.966 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.967 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2024-03-27T08:32:44.969 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-27T08:32:44.969 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:32:44.969 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.969 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:32:44.969 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:44.969 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (0 ms) 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 48 tests from LibRadosTwoPoolsPP (1 ms total) 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosTierECPP 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierECPP.Dirty 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:32:45.950 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2024-03-27T08:32:45.951 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTierECPP.HitSetNone 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosTierECPP (0 ms total) 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6267: Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.952 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (0 ms) 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.953 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (0 ms) 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.954 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2024-03-27T08:32:45.955 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2024-03-27T08:32:45.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2024-03-27T08:32:45.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (1 ms) 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.961 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2024-03-27T08:32:45.962 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6272: Skipped 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP (1 ms total) 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 77 tests from 4 test suites ran. (5991 ms total) 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 0 tests. 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 77 tests, listed below: 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierPP.Dirty 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-27T08:32:45.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2024-03-27T08:32:45.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-27T08:32:45.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-27T08:32:45.966 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2024-03-27T08:32:45.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-27T08:32:45.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_pool 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 9 tests from LibRadosPools 2024-03-27T08:32:45.969 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolList 2024-03-27T08:32:48.888 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolList (2921 ms) 2024-03-27T08:32:48.888 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolLookup 2024-03-27T08:32:51.886 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolLookup (2999 ms) 2024-03-27T08:32:51.886 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolLookup2 2024-03-27T08:32:54.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolLookup2 (3019 ms) 2024-03-27T08:32:54.907 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-27T08:32:57.881 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolLookupOtherInstance (2976 ms) 2024-03-27T08:32:57.881 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-27T08:33:00.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3061 ms) 2024-03-27T08:33:00.942 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolDelete 2024-03-27T08:33:05.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolDelete (5023 ms) 2024-03-27T08:33:05.965 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolCreateDelete 2024-03-27T08:33:10.902 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolCreateDelete (4937 ms) 2024-03-27T08:33:10.902 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-27T08:33:15.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPools.PoolCreateWithCrushRule (5003 ms) 2024-03-27T08:33:15.905 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-27T08:33:15.905 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/pool.cc:139: Skipped 2024-03-27T08:33:15.905 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 9 tests from LibRadosPools (29939 ms total) 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 9 tests from 1 test suite ran. (29939 ms total) 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 8 tests. 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-27T08:33:15.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier 2024-03-27T08:33:15.907 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:33:15.907 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:33:15.907 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_snapshots 2024-03-27T08:33:15.926 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:33:15.926 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 13 tests from 4 test suites. 2024-03-27T08:33:15.926 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:33:15.926 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosSnapshots 2024-03-27T08:33:17.885 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshots.SnapList 2024-03-27T08:33:19.919 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshots.SnapList (2034 ms) 2024-03-27T08:33:19.919 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshots.SnapRemove 2024-03-27T08:33:21.927 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshots.SnapRemove (2008 ms) 2024-03-27T08:33:21.927 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshots.Rollback 2024-03-27T08:33:23.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshots.Rollback (2007 ms) 2024-03-27T08:33:23.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshots.SnapGetName 2024-03-27T08:33:25.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshots.SnapGetName (2007 ms) 2024-03-27T08:33:26.885 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosSnapshots (8056 ms total) 2024-03-27T08:33:26.886 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:33:26.886 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-27T08:33:28.897 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-27T08:33:33.030 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Snap (4133 ms) 2024-03-27T08:33:33.030 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-27T08:33:36.994 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Rollback (3964 ms) 2024-03-27T08:33:36.995 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-27T08:33:42.062 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5068 ms) 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged (13165 ms total) 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosSnapshotsEC 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList (0 ms) 2024-03-27T08:33:42.972 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove (0 ms) 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback (0 ms) 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:33:42.973 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName (0 ms) 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosSnapshotsEC (0 ms total) 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.974 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap (0 ms) 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback (0 ms) 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC (0 ms total) 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:33:42.975 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 13 tests from 4 test suites ran. (27044 ms total) 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 7 tests. 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 6 tests, listed below: 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-27T08:33:42.976 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-27T08:33:42.977 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:33:42.977 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:33:42.977 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_snapshots_pp 2024-03-27T08:33:42.994 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:33:42.994 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 21 tests from 5 test suites. 2024-03-27T08:33:42.994 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:33:42.995 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from LibRadosSnapshotsPP 2024-03-27T08:33:42.995 INFO:tasks.workunit.client.0.smithi104.stdout:seed 96794 2024-03-27T08:33:44.978 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-27T08:33:46.889 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsPP.SnapListPP (1911 ms) 2024-03-27T08:33:46.889 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-27T08:33:48.999 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsPP.SnapRemovePP (2110 ms) 2024-03-27T08:33:48.999 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-27T08:33:51.009 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsPP.RollbackPP (2010 ms) 2024-03-27T08:33:51.009 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-27T08:33:53.017 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2008 ms) 2024-03-27T08:33:53.018 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-27T08:33:56.892 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3875 ms) 2024-03-27T08:33:57.894 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from LibRadosSnapshotsPP (11914 ms total) 2024-03-27T08:33:57.894 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:33:57.894 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-27T08:33:59.901 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-27T08:34:04.105 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4204 ms) 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/snapshots_cxx.cc:156: Skipped 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP (0 ms) 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/snapshots_cxx.cc:242: Skipped 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (0 ms) 2024-03-27T08:34:04.106 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-27T08:34:07.893 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3788 ms) 2024-03-27T08:34:07.893 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-27T08:34:10.117 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2224 ms) 2024-03-27T08:34:10.117 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-27T08:34:10.117 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-27T08:34:10.118 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:10.118 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-27T08:34:10.118 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-27T08:34:11.950 INFO:tasks.workunit.client.0.smithi104.stdout:deleting snap 9 in pool LibRadosSnapshotsSelfManagedPP_smithi104-96794-7 2024-03-27T08:34:12.955 INFO:tasks.workunit.client.0.smithi104.stdout:waiting for snaps to purge 2024-03-27T08:34:27.973 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17856 ms) 2024-03-27T08:34:28.906 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP (28072 ms total) 2024-03-27T08:34:28.906 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:28.906 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-27T08:34:28.906 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-27T08:34:28.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (6 ms) 2024-03-27T08:34:28.912 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-27T08:34:33.956 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5041 ms) 2024-03-27T08:34:33.956 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5047 ms total) 2024-03-27T08:34:33.956 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:33.956 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosSnapshotsECPP 2024-03-27T08:34:33.956 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:34:33.956 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP (0 ms) 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP (0 ms) 2024-03-27T08:34:33.957 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP (0 ms) 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP (0 ms) 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.958 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosSnapshotsECPP (0 ms total) 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP (0 ms) 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.959 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (0 ms) 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (0 ms) 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (0 ms total) 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 21 tests from 5 test suites ran. (50959 ms total) 2024-03-27T08:34:33.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 11 tests. 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 10 tests, listed below: 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-27T08:34:33.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-27T08:34:33.962 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:34:33.962 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:33.962 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_stat 2024-03-27T08:34:33.975 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:33.975 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 9 tests from 2 test suites. 2024-03-27T08:34:33.975 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:33.975 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from LibRadosStat 2024-03-27T08:34:35.931 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStat.Stat 2024-03-27T08:34:36.296 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStat.Stat (364 ms) 2024-03-27T08:34:36.296 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStat.Stat2 2024-03-27T08:34:36.321 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStat.Stat2 (26 ms) 2024-03-27T08:34:36.321 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStat.StatNS 2024-03-27T08:34:36.379 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStat.StatNS (58 ms) 2024-03-27T08:34:36.379 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStat.ClusterStat 2024-03-27T08:34:36.379 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-27T08:34:36.379 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStat.PoolStat 2024-03-27T08:34:36.390 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStat.PoolStat (11 ms) 2024-03-27T08:34:36.933 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from LibRadosStat (459 ms total) 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosStatEC 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatEC.Stat 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.Stat (0 ms) 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatEC.StatNS 2024-03-27T08:34:36.934 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.StatNS (0 ms) 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatEC.ClusterStat 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat (0 ms) 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatEC.PoolStat 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.935 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat (0 ms) 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosStatEC (0 ms total) 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 9 tests from 2 test suites ran. (2958 ms total) 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 5 tests. 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.Stat 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.StatNS 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat 2024-03-27T08:34:36.936 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat 2024-03-27T08:34:36.937 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:34:36.937 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:36.937 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_stat_pp 2024-03-27T08:34:36.953 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:36.953 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 9 tests from 2 test suites. 2024-03-27T08:34:36.953 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:36.953 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from LibRadosStatPP 2024-03-27T08:34:36.953 INFO:tasks.workunit.client.0.smithi104.stdout:seed 96928 2024-03-27T08:34:38.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatPP.StatPP 2024-03-27T08:34:39.341 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStatPP.StatPP (400 ms) 2024-03-27T08:34:39.341 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-27T08:34:39.366 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStatPP.Stat2Mtime2PP (25 ms) 2024-03-27T08:34:39.366 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-27T08:34:39.366 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-03-27T08:34:39.366 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatPP.PoolStatPP 2024-03-27T08:34:39.377 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStatPP.PoolStatPP (11 ms) 2024-03-27T08:34:39.377 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatPP.StatPPNS 2024-03-27T08:34:39.428 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosStatPP.StatPPNS (51 ms) 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from LibRadosStatPP (487 ms total) 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosStatECPP 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatECPP.StatPP 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP (0 ms) 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-27T08:34:39.944 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP (0 ms) 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosStatECPP.StatPPNS 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:34:39.945 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS (0 ms) 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosStatECPP (0 ms total) 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 9 tests from 2 test suites ran. (2990 ms total) 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 5 tests. 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP 2024-03-27T08:34:39.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS 2024-03-27T08:34:39.947 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:34:39.947 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:39.947 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_watch_notify 2024-03-27T08:34:39.962 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:39.962 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 11 tests from 2 test suites. 2024-03-27T08:34:39.962 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:39.962 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 10 tests from LibRadosWatchNotify 2024-03-27T08:34:41.899 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-27T08:34:42.248 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify_test_cb 2024-03-27T08:34:42.273 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.WatchNotify (375 ms) 2024-03-27T08:34:42.274 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-27T08:34:42.308 INFO:tasks.workunit.client.0.smithi104.stdout:waiting up to 300 for disconnect notification ... 2024-03-27T08:34:44.992 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_errcb cookie 13401280 err -107 2024-03-27T08:34:45.318 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.Watch2Delete (3044 ms) 2024-03-27T08:34:45.318 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-27T08:34:45.357 INFO:tasks.workunit.client.0.smithi104.stdout:waiting up to 300 for disconnect notification ... 2024-03-27T08:34:49.992 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_errcb cookie 13402464 err -107 2024-03-27T08:34:50.365 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete (5047 ms) 2024-03-27T08:34:50.365 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-27T08:34:50.389 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180032 cookie 13420208 2024-03-27T08:34:50.409 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2 (44 ms) 2024-03-27T08:34:50.409 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-27T08:34:50.434 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180032 cookie 13422272 2024-03-27T08:34:50.453 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.AioWatchNotify2 (45 ms) 2024-03-27T08:34:50.453 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.AioNotify 2024-03-27T08:34:50.484 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180032 cookie 13425280 2024-03-27T08:34:50.518 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.AioNotify (64 ms) 2024-03-27T08:34:50.518 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-27T08:34:50.553 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180033 cookie 13425280 2024-03-27T08:34:50.553 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180033 cookie 13432288 2024-03-27T08:34:50.580 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Multi (62 ms) 2024-03-27T08:34:50.580 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-27T08:34:50.605 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180034 cookie 13433536 2024-03-27T08:34:53.605 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180035 cookie 13433536 2024-03-27T08:34:53.627 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3047 ms) 2024-03-27T08:34:53.627 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-27T08:34:53.654 INFO:tasks.workunit.client.0.smithi104.stdout:waiting up to 1024 for osd to time us out ... 2024-03-27T08:34:57.663 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_errcb cookie 13425280 err -107 2024-03-27T08:34:58.685 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_cb from 15568 notify_id 1683627180034 cookie 13425280 2024-03-27T08:34:58.701 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.Watch3Timeout (5075 ms) 2024-03-27T08:34:58.701 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-27T08:34:58.739 INFO:tasks.workunit.client.0.smithi104.stdout:waiting up to 30 for disconnect notification ... 2024-03-27T08:34:59.995 INFO:tasks.workunit.client.0.smithi104.stdout:watch_notify2_test_errcb cookie 13425280 err -107 2024-03-27T08:35:00.748 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete2 (2046 ms) 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 10 tests from LibRadosWatchNotify (18849 ms total) 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosWatchNotifyEC 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:35:00.915 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify (0 ms) 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibRadosWatchNotifyEC (0 ms total) 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 11 tests from 2 test suites ran. (20952 ms total) 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 10 tests. 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-27T08:35:00.916 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify 2024-03-27T08:35:00.917 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:35:00.917 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:00.918 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-03-27T08:35:00.939 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:35:00.939 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify (0 ms) 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:35:00.940 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.941 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (0 ms) 2024-03-27T08:35:00.941 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:35:00.941 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:35:00.941 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP (0 ms total) 2024-03-27T08:35:00.941 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:00.941 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-27T08:35:02.907 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-27T08:35:03.256 INFO:tasks.workunit.client.0.smithi104.stdout:notify 2024-03-27T08:35:03.270 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (363 ms) 2024-03-27T08:35:03.271 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-27T08:35:03.304 INFO:tasks.workunit.client.0.smithi104.stdout:notify 2024-03-27T08:35:03.319 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (48 ms) 2024-03-27T08:35:03.319 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-27T08:35:03.354 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (35 ms) 2024-03-27T08:35:03.354 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-27T08:35:03.386 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (33 ms) 2024-03-27T08:35:03.386 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-27T08:35:03.422 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18346720 notify_id 1696512081924 notifier_gid 15571 2024-03-27T08:35:03.438 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (51 ms) 2024-03-27T08:35:03.438 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-27T08:35:03.468 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18365488 notify_id 1696512081925 notifier_gid 15571 2024-03-27T08:35:03.483 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (46 ms) 2024-03-27T08:35:03.483 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-27T08:35:03.512 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18365408 notify_id 1696512081926 notifier_gid 15571 2024-03-27T08:35:03.526 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (42 ms) 2024-03-27T08:35:03.526 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-27T08:35:03.556 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18368912 notify_id 1696512081921 notifier_gid 15571 2024-03-27T08:35:03.570 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (44 ms) 2024-03-27T08:35:03.570 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-27T08:35:03.599 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18368912 notify_id 1696512081922 notifier_gid 15571 2024-03-27T08:35:03.614 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (44 ms) 2024-03-27T08:35:03.614 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-27T08:35:03.650 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18373376 notify_id 1696512081923 notifier_gid 15571 2024-03-27T08:35:03.661 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (47 ms) 2024-03-27T08:35:03.661 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-27T08:35:03.686 INFO:tasks.workunit.client.0.smithi104.stdout: trying... 2024-03-27T08:35:03.690 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18373376 notify_id 1696512081924 notifier_gid 15571 2024-03-27T08:35:04.689 INFO:tasks.workunit.client.0.smithi104.stdout: timed out 2024-03-27T08:35:04.703 INFO:tasks.workunit.client.0.smithi104.stdout: flushing 2024-03-27T08:35:06.690 INFO:tasks.workunit.client.0.smithi104.stdout: flushed 2024-03-27T08:35:06.690 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3030 ms) 2024-03-27T08:35:06.691 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-27T08:35:06.723 INFO:tasks.workunit.client.0.smithi104.stdout: trying... 2024-03-27T08:35:06.727 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18373376 notify_id 1696512081923 notifier_gid 15571 2024-03-27T08:35:07.727 INFO:tasks.workunit.client.0.smithi104.stdout: timed out 2024-03-27T08:35:07.741 INFO:tasks.workunit.client.0.smithi104.stdout: flushing 2024-03-27T08:35:09.728 INFO:tasks.workunit.client.0.smithi104.stdout: flushed 2024-03-27T08:35:09.728 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3037 ms) 2024-03-27T08:35:09.728 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-27T08:35:09.759 INFO:tasks.workunit.client.0.smithi104.stdout:List watches 2024-03-27T08:35:09.759 INFO:tasks.workunit.client.0.smithi104.stdout:notify2 2024-03-27T08:35:09.763 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18373376 notify_id 1696512081923 notifier_gid 15571 2024-03-27T08:35:12.769 INFO:tasks.workunit.client.0.smithi104.stdout:notify2 done 2024-03-27T08:35:12.770 INFO:tasks.workunit.client.0.smithi104.stdout:watch_check 2024-03-27T08:35:12.770 INFO:tasks.workunit.client.0.smithi104.stdout:unwatch2 2024-03-27T08:35:12.782 INFO:tasks.workunit.client.0.smithi104.stdout: flushing 2024-03-27T08:35:12.782 INFO:tasks.workunit.client.0.smithi104.stdout:done 2024-03-27T08:35:12.782 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3054 ms) 2024-03-27T08:35:12.782 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-27T08:35:12.813 INFO:tasks.workunit.client.0.smithi104.stdout:List watches 2024-03-27T08:35:12.813 INFO:tasks.workunit.client.0.smithi104.stdout:notify2 2024-03-27T08:35:12.817 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 18373376 notify_id 1700807049221 notifier_gid 15571 2024-03-27T08:35:15.823 INFO:tasks.workunit.client.0.smithi104.stdout:notify2 done 2024-03-27T08:35:15.824 INFO:tasks.workunit.client.0.smithi104.stdout:watch_check 2024-03-27T08:35:15.824 INFO:tasks.workunit.client.0.smithi104.stdout:unwatch2 2024-03-27T08:35:15.837 INFO:tasks.workunit.client.0.smithi104.stdout: flushing 2024-03-27T08:35:15.837 INFO:tasks.workunit.client.0.smithi104.stdout:done 2024-03-27T08:35:15.837 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3055 ms) 2024-03-27T08:35:15.930 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (12929 ms total) 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 16 tests from 2 test suites ran. (14991 ms total) 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 14 tests. 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 2 tests, listed below: 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-27T08:35:15.931 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-27T08:35:15.932 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:35:15.932 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:15.932 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_cmd 2024-03-27T08:35:15.958 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:35:15.958 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-27T08:35:15.958 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:15.958 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosCmd 2024-03-27T08:35:15.958 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.MonDescribe 2024-03-27T08:35:16.001 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.MonDescribe (43 ms) 2024-03-27T08:35:16.001 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.OSDCmd 2024-03-27T08:35:16.043 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.OSDCmd (43 ms) 2024-03-27T08:35:16.044 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.PGCmd 2024-03-27T08:35:18.949 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.PGCmd (2905 ms) 2024-03-27T08:35:18.950 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.WatchLog 2024-03-27T08:35:18.973 INFO:tasks.workunit.client.0.smithi104.stdout:got: 2024-03-27T08:35:17.913166+0000 mon.a [INF] from='client.? 172.21.15.104:0/653794363' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi104-96985-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-27T08:35:19.947 INFO:tasks.workunit.client.0.smithi104.stdout:got: 2024-03-27T08:35:18.982388+0000 client.admin [INF] onexx 2024-03-27T08:35:19.947 INFO:tasks.workunit.client.0.smithi104.stdout:got: 2024-03-27T08:35:18.982517+0000 mon.a [INF] from='client.? 172.21.15.104:0/3347455166' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-27T08:35:23.954 INFO:tasks.workunit.client.0.smithi104.stdout:got: 2024-03-27T08:35:21.960568+0000 mon.a [INF] from='client.? 172.21.15.104:0/3347455166' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-27T08:35:23.961 INFO:tasks.workunit.client.0.smithi104.stdout:got: 2024-03-27T08:35:23.961253+0000 client.admin [INF] threexx 2024-03-27T08:35:23.961 INFO:tasks.workunit.client.0.smithi104.stdout:got: 2024-03-27T08:35:23.961525+0000 mon.a [INF] from='client.? 172.21.15.104:0/3347455166' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-27T08:35:27.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.WatchLog (9019 ms) 2024-03-27T08:35:27.968 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosCmd (12010 ms total) 2024-03-27T08:35:27.968 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:27.968 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:27.968 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 4 tests from 1 test suite ran. (12010 ms total) 2024-03-27T08:35:27.968 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 4 tests. 2024-03-27T08:35:27.970 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:35:27.970 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:27.970 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_cmd_pp 2024-03-27T08:35:27.991 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:35:27.991 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-27T08:35:27.991 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:27.992 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosCmd 2024-03-27T08:35:27.992 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.MonDescribePP 2024-03-27T08:35:28.023 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.MonDescribePP (31 ms) 2024-03-27T08:35:28.023 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.OSDCmdPP 2024-03-27T08:35:28.055 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.OSDCmdPP (32 ms) 2024-03-27T08:35:28.055 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCmd.PGCmdPP 2024-03-27T08:35:30.144 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCmd.PGCmdPP (2089 ms) 2024-03-27T08:35:30.144 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from LibRadosCmd (2152 ms total) 2024-03-27T08:35:30.145 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:30.145 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:30.145 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 3 tests from 1 test suite ran. (2152 ms total) 2024-03-27T08:35:30.145 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 3 tests. 2024-03-27T08:35:30.146 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:35:30.146 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:30.146 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_service 2024-03-27T08:35:30.174 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:35:30.182+0000 7fc90d660840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:35:30.175 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:35:30.182+0000 7fc90d660840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:35:30.175 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:35:30.183+0000 7fc90d660840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:35:30.175 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-27T08:35:30.175 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:30.175 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosService 2024-03-27T08:35:30.175 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosService.RegisterEarly 2024-03-27T08:35:35.211 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosService.RegisterEarly (5037 ms) 2024-03-27T08:35:35.212 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosService.RegisterLate 2024-03-27T08:35:35.250 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosService.RegisterLate (38 ms) 2024-03-27T08:35:35.250 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosService.StatusFormat 2024-03-27T08:35:35.395 INFO:tasks.workunit.client.0.smithi104.stdout: cluster: 2024-03-27T08:35:35.395 INFO:tasks.workunit.client.0.smithi104.stdout: id: 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:35:35.395 INFO:tasks.workunit.client.0.smithi104.stdout: health: HEALTH_WARN 2024-03-27T08:35:35.395 INFO:tasks.workunit.client.0.smithi104.stdout: 1 pool(s) do not have an application enabled 2024-03-27T08:35:35.395 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:35.395 INFO:tasks.workunit.client.0.smithi104.stdout: services: 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: mon: 1 daemons, quorum a (age 9m) 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: mgr: x(active, since 9m) 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: osd: 4 osds: 4 up (since 9m), 4 in (since 9m) 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: laundry: 1 daemon active (1 hosts) 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: data: 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: pools: 3 pools, 41 pgs 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: objects: 3 objects, 577 KiB 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: usage: 168 MiB used, 400 GiB / 400 GiB avail 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: pgs: 41 active+clean 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:35.396 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:37.426 INFO:tasks.workunit.client.0.smithi104.stdout: cluster: 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: id: 756f8b0d-eb2f-4187-9610-708a3d134344 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: health: HEALTH_WARN 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: 1 pool(s) do not have an application enabled 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: services: 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: mon: 1 daemons, quorum a (age 9m) 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: mgr: x(active, since 9m) 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: osd: 4 osds: 4 up (since 9m), 4 in (since 9m) 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: foo: 16 portals active (1 hosts, 3 zones) 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: data: 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: pools: 3 pools, 41 pgs 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: objects: 3 objects, 577 KiB 2024-03-27T08:35:37.427 INFO:tasks.workunit.client.0.smithi104.stdout: usage: 168 MiB used, 400 GiB / 400 GiB avail 2024-03-27T08:35:37.428 INFO:tasks.workunit.client.0.smithi104.stdout: pgs: 41 active+clean 2024-03-27T08:35:37.428 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:37.428 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:37.449 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosService.StatusFormat (2199 ms) 2024-03-27T08:35:37.449 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosService.Status 2024-03-27T08:35:57.496 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosService.Status (20047 ms) 2024-03-27T08:35:57.496 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosService (27321 ms total) 2024-03-27T08:35:57.496 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:35:57.496 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:57.496 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 4 tests from 1 test suite ran. (27321 ms total) 2024-03-27T08:35:57.496 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 4 tests. 2024-03-27T08:35:57.501 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:35:57.501 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:57.501 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_service_pp 2024-03-27T08:35:57.534 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:35:57.542+0000 7f7371142840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:35:57.534 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:35:57.542+0000 7f7371142840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:35:57.534 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:35:57.542+0000 7f7371142840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:35:57.535 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-27T08:35:57.535 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:57.535 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosServicePP 2024-03-27T08:35:57.535 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosServicePP.RegisterEarly 2024-03-27T08:36:02.573 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosServicePP.RegisterEarly (5040 ms) 2024-03-27T08:36:02.574 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosServicePP.RegisterLate 2024-03-27T08:36:02.628 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosServicePP.RegisterLate (54 ms) 2024-03-27T08:36:02.628 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosServicePP.Status 2024-03-27T08:36:22.668 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosServicePP.Status (20040 ms) 2024-03-27T08:36:22.669 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosServicePP.Close 2024-03-27T08:36:22.669 INFO:tasks.workunit.client.0.smithi104.stdout:attempt 0 of 20 2024-03-27T08:36:27.734 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosServicePP.Close (5066 ms) 2024-03-27T08:36:27.734 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 4 tests from LibRadosServicePP (30200 ms total) 2024-03-27T08:36:27.735 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:36:27.735 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:36:27.735 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 4 tests from 1 test suite ran. (30200 ms total) 2024-03-27T08:36:27.735 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 4 tests. 2024-03-27T08:36:27.736 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:36:27.736 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:36:27.737 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_c_write_operations 2024-03-27T08:36:27.757 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:36:27.757 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 8 tests from 2 test suites. 2024-03-27T08:36:27.757 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:36:27.757 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibradosCWriteOps 2024-03-27T08:36:27.757 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibradosCWriteOps.NewDelete 2024-03-27T08:36:27.757 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-27T08:36:27.758 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-27T08:36:27.758 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:36:27.758 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from LibRadosCWriteOps 2024-03-27T08:36:27.758 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.assertExists 2024-03-27T08:36:29.977 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.assertExists (2220 ms) 2024-03-27T08:36:29.977 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-27T08:36:32.992 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3015 ms) 2024-03-27T08:36:32.993 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.Xattrs 2024-03-27T08:36:36.011 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.Xattrs (3019 ms) 2024-03-27T08:36:36.011 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.Write 2024-03-27T08:36:39.023 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.Write (3012 ms) 2024-03-27T08:36:39.023 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.Exec 2024-03-27T08:36:42.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.Exec (3011 ms) 2024-03-27T08:36:42.034 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.WriteSame 2024-03-27T08:36:45.054 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.WriteSame (3020 ms) 2024-03-27T08:36:45.054 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] LibRadosCWriteOps.CmpExt 2024-03-27T08:36:55.408 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] LibRadosCWriteOps.CmpExt (10354 ms) 2024-03-27T08:36:55.409 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from LibRadosCWriteOps (27651 ms total) 2024-03-27T08:36:55.409 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:36:55.409 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:36:55.409 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 8 tests from 2 test suites ran. (27651 ms total) 2024-03-27T08:36:55.409 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 8 tests. 2024-03-27T08:36:55.410 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:36:55.410 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:36:55.410 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_c_read_operations 2024-03-27T08:36:55.427 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:36:55.428 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 17 tests from 1 test suite. 2024-03-27T08:36:55.428 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:36:55.428 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 17 tests from CReadOpsTest 2024-03-27T08:36:57.417 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.NewDelete 2024-03-27T08:36:57.418 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-27T08:36:57.418 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.SetOpFlags 2024-03-27T08:36:57.861 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.SetOpFlags (444 ms) 2024-03-27T08:36:57.861 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.AssertExists 2024-03-27T08:36:57.897 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.AssertExists (37 ms) 2024-03-27T08:36:57.897 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.AssertVersion 2024-03-27T08:36:57.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.AssertVersion (41 ms) 2024-03-27T08:36:57.939 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.CmpXattr 2024-03-27T08:36:58.110 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.CmpXattr (172 ms) 2024-03-27T08:36:58.111 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.Read 2024-03-27T08:36:58.157 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.Read (46 ms) 2024-03-27T08:36:58.157 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.Checksum 2024-03-27T08:36:58.201 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.Checksum (45 ms) 2024-03-27T08:36:58.201 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.RWOrderedRead 2024-03-27T08:36:58.232 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.RWOrderedRead (31 ms) 2024-03-27T08:36:58.233 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.ShortRead 2024-03-27T08:36:58.271 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.ShortRead (38 ms) 2024-03-27T08:36:58.271 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.Exec 2024-03-27T08:36:58.296 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.Exec (26 ms) 2024-03-27T08:36:58.296 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.ExecUserBuf 2024-03-27T08:36:58.333 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.ExecUserBuf (36 ms) 2024-03-27T08:36:58.333 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.Stat 2024-03-27T08:36:58.369 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.Stat (36 ms) 2024-03-27T08:36:58.369 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.Stat2 2024-03-27T08:36:58.404 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.Stat2 (35 ms) 2024-03-27T08:36:58.404 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.Omap 2024-03-27T08:36:58.597 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.Omap (194 ms) 2024-03-27T08:36:58.597 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.OmapNuls 2024-03-27T08:36:58.680 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.OmapNuls (82 ms) 2024-03-27T08:36:58.680 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.GetXattrs 2024-03-27T08:36:58.748 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.GetXattrs (68 ms) 2024-03-27T08:36:58.748 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] CReadOpsTest.CmpExt 2024-03-27T08:36:58.781 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] CReadOpsTest.CmpExt (33 ms) 2024-03-27T08:36:59.425 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 17 tests from CReadOpsTest (1364 ms total) 2024-03-27T08:36:59.425 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:36:59.426 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:36:59.426 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 17 tests from 1 test suite ran. (3997 ms total) 2024-03-27T08:36:59.426 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 17 tests. 2024-03-27T08:36:59.427 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:36:59.427 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:36:59.427 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-03-27T08:36:59.458 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:36:59.466+0000 7fe52d25d6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:36:59.458 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:36:59.466+0000 7fe52d25d6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:36:59.458 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:36:59.466+0000 7fe52d25d6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:36:59.459 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 1 test from 1 test suite. 2024-03-27T08:36:59.459 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:36:59.459 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from ClsTestRemoteReads 2024-03-27T08:36:59.459 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] ClsTestRemoteReads.TestGather 2024-03-27T08:36:59.459 INFO:tasks.workunit.client.0.smithi104.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/cls_remote_reads.cc:13: Skipped 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] ClsTestRemoteReads.TestGather (0 ms) 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from ClsTestRemoteReads (0 ms total) 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 1 test from 1 test suite ran. (0 ms total) 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 0 tests. 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-27T08:36:59.460 INFO:tasks.workunit.client.0.smithi104.stdout:[ SKIPPED ] ClsTestRemoteReads.TestGather 2024-03-27T08:36:59.461 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:36:59.461 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:36:59.461 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_list_parallel 2024-03-27T08:37:02.422 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: starting. 2024-03-27T08:37:02.422 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: creating pool ceph_test_rados_list_parallel.smithi104-97668 2024-03-27T08:37:02.422 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: created object 0... 2024-03-27T08:37:02.423 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: created object 25... 2024-03-27T08:37:02.423 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: created object 49... 2024-03-27T08:37:02.423 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: finishing. 2024-03-27T08:37:02.423 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97669]: shutting down. 2024-03-27T08:37:02.849 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97670]: starting. 2024-03-27T08:37:02.849 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97670]: listing objects. 2024-03-27T08:37:02.849 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97670]: listed object 0... 2024-03-27T08:37:02.849 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97670]: listed object 25... 2024-03-27T08:37:02.850 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97670]: saw 50 objects 2024-03-27T08:37:02.850 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97670]: shutting down. 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: starting. 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: creating pool ceph_test_rados_list_parallel.smithi104-97668 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: created object 0... 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: created object 25... 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: created object 49... 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: finishing. 2024-03-27T08:37:03.405 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97705]: shutting down. 2024-03-27T08:37:03.674 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:03.675 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97706]: starting. 2024-03-27T08:37:03.675 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97706]: listing objects. 2024-03-27T08:37:03.675 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97706]: listed object 0... 2024-03-27T08:37:03.675 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97706]: listed object 25... 2024-03-27T08:37:03.675 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97706]: saw 43 objects 2024-03-27T08:37:03.675 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97706]: shutting down. 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[97707]: starting. 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[97707]: removed 25 objects... 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[97707]: removed half of the objects 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[97707]: removed 50 objects... 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[97707]: removed 50 objects 2024-03-27T08:37:03.923 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[97707]: shutting down. 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: starting. 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: creating pool ceph_test_rados_list_parallel.smithi104-97668 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: created object 0... 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: created object 25... 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: created object 49... 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: finishing. 2024-03-27T08:37:04.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_6_[97756]: shutting down. 2024-03-27T08:37:04.778 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:04.778 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:04.778 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: starting. 2024-03-27T08:37:04.779 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: listing objects. 2024-03-27T08:37:04.779 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: listed object 0... 2024-03-27T08:37:04.779 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: listed object 25... 2024-03-27T08:37:04.779 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: listed object 50... 2024-03-27T08:37:04.779 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: saw 56 objects 2024-03-27T08:37:04.779 INFO:tasks.workunit.client.0.smithi104.stdout:process_7_[97757]: shutting down. 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:process_8_[97758]: starting. 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:process_8_[97758]: added 25 objects... 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:process_8_[97758]: added half of the objects 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:process_8_[97758]: added 50 objects... 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:process_8_[97758]: added 50 objects 2024-03-27T08:37:05.050 INFO:tasks.workunit.client.0.smithi104.stdout:process_8_[97758]: shutting down. 2024-03-27T08:37:05.669 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: starting. 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: creating pool ceph_test_rados_list_parallel.smithi104-97668 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: created object 0... 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: created object 25... 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: created object 49... 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: finishing. 2024-03-27T08:37:05.670 INFO:tasks.workunit.client.0.smithi104.stdout:process_9_[97807]: shutting down. 2024-03-27T08:37:06.014 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.014 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: starting. 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: listing objects. 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: listed object 0... 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: listed object 25... 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: listed object 50... 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: listed object 75... 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: saw 98 objects 2024-03-27T08:37:06.015 INFO:tasks.workunit.client.0.smithi104.stdout:process_10_[97808]: shutting down. 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:process_13_[97811]: starting. 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:process_13_[97811]: removed 25 objects... 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:process_13_[97811]: removed half of the objects 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:process_13_[97811]: removed 50 objects... 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:process_13_[97811]: removed 50 objects 2024-03-27T08:37:06.303 INFO:tasks.workunit.client.0.smithi104.stdout:process_13_[97811]: shutting down. 2024-03-27T08:37:06.357 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.357 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_11_[97809]: starting. 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_11_[97809]: added 25 objects... 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_11_[97809]: added half of the objects 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_11_[97809]: added 50 objects... 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_11_[97809]: added 50 objects 2024-03-27T08:37:06.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_11_[97809]: shutting down. 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_12_[97810]: starting. 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_12_[97810]: added 25 objects... 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_12_[97810]: added half of the objects 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_12_[97810]: added 50 objects... 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_12_[97810]: added 50 objects 2024-03-27T08:37:06.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_12_[97810]: shutting down. 2024-03-27T08:37:06.934 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.934 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.934 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.934 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: starting. 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: creating pool ceph_test_rados_list_parallel.smithi104-97668 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: created object 0... 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: created object 25... 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: created object 49... 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: finishing. 2024-03-27T08:37:06.935 INFO:tasks.workunit.client.0.smithi104.stdout:process_14_[97892]: shutting down. 2024-03-27T08:37:07.238 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: starting. 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listing objects. 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listed object 0... 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listed object 25... 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listed object 50... 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listed object 75... 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listed object 100... 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: listed object 125... 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: saw 150 objects 2024-03-27T08:37:07.239 INFO:tasks.workunit.client.0.smithi104.stdout:process_15_[97893]: shutting down. 2024-03-27T08:37:07.506 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.506 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.506 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.506 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:07.506 INFO:tasks.workunit.client.0.smithi104.stdout:process_16_[97894]: starting. 2024-03-27T08:37:07.506 INFO:tasks.workunit.client.0.smithi104.stdout:process_16_[97894]: added 25 objects... 2024-03-27T08:37:07.507 INFO:tasks.workunit.client.0.smithi104.stdout:process_16_[97894]: added half of the objects 2024-03-27T08:37:07.507 INFO:tasks.workunit.client.0.smithi104.stdout:process_16_[97894]: added 50 objects... 2024-03-27T08:37:07.507 INFO:tasks.workunit.client.0.smithi104.stdout:process_16_[97894]: added 50 objects 2024-03-27T08:37:07.507 INFO:tasks.workunit.client.0.smithi104.stdout:process_16_[97894]: shutting down. 2024-03-27T08:37:08.450 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:08.450 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stdout:******* SUCCESS ********** 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:37:08.451 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_open_pools_parallel 2024-03-27T08:37:11.357 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: starting. 2024-03-27T08:37:11.357 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: creating pool ceph_test_rados_open_pools_parallel.smithi104-97959 2024-03-27T08:37:11.357 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: created object 0... 2024-03-27T08:37:11.357 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: created object 25... 2024-03-27T08:37:11.357 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: created object 49... 2024-03-27T08:37:11.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: finishing. 2024-03-27T08:37:11.358 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[97960]: shutting down. 2024-03-27T08:37:11.467 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97961]: starting. 2024-03-27T08:37:11.468 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97961]: rados_pool_create. 2024-03-27T08:37:11.468 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97961]: rados_ioctx_create. 2024-03-27T08:37:11.468 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[97961]: shutting down. 2024-03-27T08:37:14.370 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:14.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: starting. 2024-03-27T08:37:14.370 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: creating pool ceph_test_rados_open_pools_parallel.smithi104-97959 2024-03-27T08:37:14.371 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: created object 0... 2024-03-27T08:37:14.371 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: created object 25... 2024-03-27T08:37:14.371 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: created object 49... 2024-03-27T08:37:14.371 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: finishing. 2024-03-27T08:37:14.371 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[97996]: shutting down. 2024-03-27T08:37:14.479 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:14.479 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97997]: starting. 2024-03-27T08:37:14.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97997]: rados_pool_create. 2024-03-27T08:37:14.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97997]: rados_ioctx_create. 2024-03-27T08:37:14.480 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[97997]: shutting down. 2024-03-27T08:37:14.481 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:14.481 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:14.481 INFO:tasks.workunit.client.0.smithi104.stdout:******* SUCCESS ********** 2024-03-27T08:37:14.481 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:37:14.482 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:37:14.482 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_rados_delete_pools_parallel 2024-03-27T08:37:17.410 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: starting. 2024-03-27T08:37:17.410 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: creating pool ceph_test_rados_delete_pools_parallel.smithi104-98032 2024-03-27T08:37:17.410 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: created object 0... 2024-03-27T08:37:17.410 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: created object 25... 2024-03-27T08:37:17.410 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: created object 49... 2024-03-27T08:37:17.410 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: finishing. 2024-03-27T08:37:17.411 INFO:tasks.workunit.client.0.smithi104.stdout:process_1_[98033]: shutting down. 2024-03-27T08:37:17.490 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[98034]: starting. 2024-03-27T08:37:17.490 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[98034]: deleting pool ceph_test_rados_delete_pools_parallel.smithi104-98032 2024-03-27T08:37:17.490 INFO:tasks.workunit.client.0.smithi104.stdout:process_2_[98034]: shutting down. 2024-03-27T08:37:20.424 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: starting. 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: creating pool ceph_test_rados_delete_pools_parallel.smithi104-98032 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: created object 0... 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: created object 25... 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: created object 49... 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: finishing. 2024-03-27T08:37:20.425 INFO:tasks.workunit.client.0.smithi104.stdout:process_3_[98069]: shutting down. 2024-03-27T08:37:20.537 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:20.538 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[98071]: starting. 2024-03-27T08:37:20.538 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[98071]: listing objects. 2024-03-27T08:37:20.538 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[98071]: listed object 0... 2024-03-27T08:37:20.538 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[98071]: listed object 25... 2024-03-27T08:37:20.538 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[98071]: saw 50 objects 2024-03-27T08:37:20.538 INFO:tasks.workunit.client.0.smithi104.stdout:process_5_[98071]: shutting down. 2024-03-27T08:37:21.511 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:21.511 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[98070]: starting. 2024-03-27T08:37:21.512 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[98070]: deleting pool ceph_test_rados_delete_pools_parallel.smithi104-98032 2024-03-27T08:37:21.512 INFO:tasks.workunit.client.0.smithi104.stdout:process_4_[98070]: shutting down. 2024-03-27T08:37:21.513 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:21.513 INFO:tasks.workunit.client.0.smithi104.stdout:******************************* 2024-03-27T08:37:21.513 INFO:tasks.workunit.client.0.smithi104.stdout:******* SUCCESS ********** 2024-03-27T08:37:21.514 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:37:21.514 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:37:21.514 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:37:21.515 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' cls = ec_io ']' 2024-03-27T08:37:21.515 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' cls = ec_list ']' 2024-03-27T08:37:21.515 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_cls 2024-03-27T08:37:21.540 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:37:21.540 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 2 tests from 1 test suite. 2024-03-27T08:37:21.540 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:37:21.540 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from NeoRadosCls 2024-03-27T08:37:21.540 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosCls.DNE 2024-03-27T08:37:24.528 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosCls.DNE (2988 ms) 2024-03-27T08:37:24.528 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosCls.RemoteReads 2024-03-27T08:37:25.530 INFO:tasks.workunit.client.0.smithi104.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:37:26.534 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosCls.RemoteReads (2006 ms) 2024-03-27T08:37:26.535 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from NeoRadosCls (4994 ms total) 2024-03-27T08:37:26.535 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:37:26.535 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:37:26.535 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 2 tests from 1 test suite ran. (4994 ms total) 2024-03-27T08:37:26.535 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 2 tests. 2024-03-27T08:37:26.536 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:37:26.537 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:37:26.537 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:37:26.537 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' cmd = ec_io ']' 2024-03-27T08:37:26.537 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' cmd = ec_list ']' 2024-03-27T08:37:26.537 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_cmd 2024-03-27T08:37:26.560 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:37:26.560 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-27T08:37:26.560 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:37:26.560 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from NeoRadosCmd 2024-03-27T08:37:26.560 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosCmd.MonDescribe 2024-03-27T08:37:28.544 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosCmd.MonDescribe (1984 ms) 2024-03-27T08:37:28.544 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosCmd.OSDCmd 2024-03-27T08:37:30.551 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosCmd.OSDCmd (2008 ms) 2024-03-27T08:37:30.552 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosCmd.PGCmd 2024-03-27T08:37:33.563 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosCmd.PGCmd (3011 ms) 2024-03-27T08:37:33.563 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from NeoRadosCmd (7003 ms total) 2024-03-27T08:37:33.563 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:37:33.563 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:37:33.563 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 3 tests from 1 test suite ran. (7003 ms total) 2024-03-27T08:37:33.563 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 3 tests. 2024-03-27T08:37:33.565 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:37:33.565 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:37:33.565 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:37:33.565 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' handler_error = ec_io ']' 2024-03-27T08:37:33.565 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' handler_error = ec_list ']' 2024-03-27T08:37:33.565 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_handler_error 2024-03-27T08:37:33.588 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:37:33.588 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 1 test from 1 test suite. 2024-03-27T08:37:33.588 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:37:33.588 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from neocls_handler_error 2024-03-27T08:37:33.588 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] neocls_handler_error.test_handler_error 2024-03-27T08:37:36.577 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] neocls_handler_error.test_handler_error (2989 ms) 2024-03-27T08:37:36.577 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 1 test from neocls_handler_error (2989 ms total) 2024-03-27T08:37:36.577 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:37:36.577 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:37:36.577 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 1 test from 1 test suite ran. (2989 ms total) 2024-03-27T08:37:36.577 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 1 test. 2024-03-27T08:37:36.579 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:37:36.579 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:37:36.579 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:37:36.579 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' io = ec_io ']' 2024-03-27T08:37:36.579 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' io = ec_list ']' 2024-03-27T08:37:36.579 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_io 2024-03-27T08:37:36.598 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:37:36.598 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 14 tests from 1 test suite. 2024-03-27T08:37:36.599 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:37:36.599 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from NeoRadosIo 2024-03-27T08:37:36.599 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.Limits 2024-03-27T08:37:36.939 INFO:tasks.workunit.client.0.smithi104.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:37:37.945 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.Limits (1346 ms) 2024-03-27T08:37:37.945 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.SimpleWrite 2024-03-27T08:37:40.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.SimpleWrite (3022 ms) 2024-03-27T08:37:40.967 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.ReadOp 2024-03-27T08:37:43.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.ReadOp (2997 ms) 2024-03-27T08:37:43.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.SparseRead 2024-03-27T08:37:46.984 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.SparseRead (3020 ms) 2024-03-27T08:37:46.984 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.RoundTrip 2024-03-27T08:37:49.988 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.RoundTrip (3004 ms) 2024-03-27T08:37:49.988 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-27T08:37:53.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.ReadIntoBuufferlist (3045 ms) 2024-03-27T08:37:53.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-27T08:37:56.011 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.OverlappingWriteRoundTrip (2979 ms) 2024-03-27T08:37:56.012 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-27T08:37:59.025 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.WriteFullRoundTrip (3013 ms) 2024-03-27T08:37:59.025 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-27T08:38:01.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.AppendRoundTrip (2921 ms) 2024-03-27T08:38:01.946 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.Trunc 2024-03-27T08:38:05.038 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.Trunc (3092 ms) 2024-03-27T08:38:05.038 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.Remove 2024-03-27T08:38:08.049 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.Remove (3011 ms) 2024-03-27T08:38:08.049 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-27T08:38:11.065 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.XattrsRoundTrip (3017 ms) 2024-03-27T08:38:11.065 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.RmXattr 2024-03-27T08:38:14.072 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.RmXattr (3006 ms) 2024-03-27T08:38:14.072 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosIo.GetXattrs 2024-03-27T08:38:17.095 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosIo.GetXattrs (3024 ms) 2024-03-27T08:38:17.096 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from NeoRadosIo (40497 ms total) 2024-03-27T08:38:17.096 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:38:17.096 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:38:17.096 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 14 tests from 1 test suite ran. (40497 ms total) 2024-03-27T08:38:17.096 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 14 tests. 2024-03-27T08:38:17.100 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:38:17.100 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:17.100 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:38:17.100 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' ec_io = ec_io ']' 2024-03-27T08:38:17.100 INFO:tasks.workunit.client.0.smithi104.stderr:+ echo 'Skipping EC with Crimson' 2024-03-27T08:38:17.100 INFO:tasks.workunit.client.0.smithi104.stdout:Skipping EC with Crimson 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ continue 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' list = ec_io ']' 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' list = ec_list ']' 2024-03-27T08:38:17.101 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_list 2024-03-27T08:38:17.119 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:38:17.119 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-27T08:38:17.119 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:38:17.119 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from NeoradosList 2024-03-27T08:38:17.119 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoradosList.ListObjects 2024-03-27T08:38:20.104 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoradosList.ListObjects (2987 ms) 2024-03-27T08:38:20.104 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoradosList.ListObjectsNS 2024-03-27T08:38:23.123 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoradosList.ListObjectsNS (3019 ms) 2024-03-27T08:38:23.123 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoradosList.ListObjectsMany 2024-03-27T08:38:32.429 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoradosList.ListObjectsMany (9306 ms) 2024-03-27T08:38:32.430 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 3 tests from NeoradosList (15312 ms total) 2024-03-27T08:38:32.430 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:38:32.430 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:38:32.430 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 3 tests from 1 test suite ran. (15312 ms total) 2024-03-27T08:38:32.430 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 3 tests. 2024-03-27T08:38:32.432 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:38:32.432 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:32.432 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:38:32.432 INFO:tasks.workunit.client.0.smithi104.stdout:Skipping EC with Crimson 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' ec_list = ec_io ']' 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' ec_list = ec_list ']' 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ echo 'Skipping EC with Crimson' 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ continue 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' misc = ec_io ']' 2024-03-27T08:38:32.433 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' misc = ec_list ']' 2024-03-27T08:38:32.434 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_misc 2024-03-27T08:38:32.451 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:38:32.452 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 12 tests from 1 test suite. 2024-03-27T08:38:32.452 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:38:32.452 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 12 tests from NeoRadosMisc 2024-03-27T08:38:32.452 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.Version 2024-03-27T08:38:34.435 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.Version (1983 ms) 2024-03-27T08:38:34.435 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-27T08:38:36.452 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.WaitOSDMap (2017 ms) 2024-03-27T08:38:36.452 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.LongName 2024-03-27T08:38:38.959 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.LongName (2508 ms) 2024-03-27T08:38:38.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.LongLocator 2024-03-27T08:38:41.971 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.LongLocator (3012 ms) 2024-03-27T08:38:41.972 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.LongNamespace 2024-03-27T08:38:44.987 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.LongNamespace (3015 ms) 2024-03-27T08:38:44.987 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.LongAttrName 2024-03-27T08:38:48.000 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.LongAttrName (3013 ms) 2024-03-27T08:38:48.000 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.Exec 2024-03-27T08:38:51.018 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.Exec (3019 ms) 2024-03-27T08:38:51.019 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.Operate1 2024-03-27T08:38:54.029 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.Operate1 (3010 ms) 2024-03-27T08:38:54.029 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.Operate2 2024-03-27T08:38:56.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.Operate2 (2931 ms) 2024-03-27T08:38:56.960 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.BigObject 2024-03-27T08:38:59.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.BigObject (3002 ms) 2024-03-27T08:38:59.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.BigAttr 2024-03-27T08:39:01.978 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.BigAttr (2015 ms) 2024-03-27T08:39:01.978 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosMisc.WriteSame 2024-03-27T08:39:02.982 INFO:tasks.workunit.client.0.smithi104.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:39:03.984 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosMisc.WriteSame (2006 ms) 2024-03-27T08:39:03.984 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 12 tests from NeoRadosMisc (31532 ms total) 2024-03-27T08:39:03.984 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:39:03.984 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:39:03.984 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 12 tests from 1 test suite ran. (31532 ms total) 2024-03-27T08:39:03.984 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 12 tests. 2024-03-27T08:39:03.987 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:39:03.987 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:39:03.987 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:39:03.987 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' pool = ec_io ']' 2024-03-27T08:39:03.987 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' pool = ec_list ']' 2024-03-27T08:39:03.988 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_pool 2024-03-27T08:39:04.012 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:39:04.013 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 6 tests from 1 test suite. 2024-03-27T08:39:04.013 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:39:04.013 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 6 tests from NeoRadosPools 2024-03-27T08:39:04.013 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosPools.PoolList 2024-03-27T08:39:05.992 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosPools.PoolList (1980 ms) 2024-03-27T08:39:05.992 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosPools.PoolLookup 2024-03-27T08:39:07.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosPools.PoolLookup (1968 ms) 2024-03-27T08:39:07.961 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-27T08:39:10.010 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosPools.PoolLookupOtherInstance (2049 ms) 2024-03-27T08:39:10.010 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosPools.PoolDelete 2024-03-27T08:39:14.023 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosPools.PoolDelete (4013 ms) 2024-03-27T08:39:14.023 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-27T08:39:16.031 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosPools.PoolCreateDelete (2008 ms) 2024-03-27T08:39:16.031 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-27T08:39:17.963 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosPools.PoolCreateWithCrushRule (1933 ms) 2024-03-27T08:39:17.963 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 6 tests from NeoRadosPools (13951 ms total) 2024-03-27T08:39:17.964 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:39:17.964 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:39:17.964 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 6 tests from 1 test suite ran. (13951 ms total) 2024-03-27T08:39:17.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 6 tests. 2024-03-27T08:39:17.966 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:39:17.966 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:39:17.966 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:39:17.966 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' read_operations = ec_io ']' 2024-03-27T08:39:17.966 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' read_operations = ec_list ']' 2024-03-27T08:39:17.966 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_read_operations 2024-03-27T08:39:17.991 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:39:17.991 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 14 tests from 1 test suite. 2024-03-27T08:39:17.991 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:39:17.991 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from NeoRadosReadOps 2024-03-27T08:39:17.991 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-27T08:39:20.977 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.SetOpFlags (2987 ms) 2024-03-27T08:39:20.978 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.AssertExists 2024-03-27T08:39:23.985 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.AssertExists (3008 ms) 2024-03-27T08:39:23.985 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.AssertVersion 2024-03-27T08:39:26.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.AssertVersion (2979 ms) 2024-03-27T08:39:26.964 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.CmpXattr 2024-03-27T08:39:30.009 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.CmpXattr (3044 ms) 2024-03-27T08:39:30.010 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.Read 2024-03-27T08:39:33.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.Read (3024 ms) 2024-03-27T08:39:33.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.Checksum 2024-03-27T08:39:36.042 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.Checksum (3009 ms) 2024-03-27T08:39:36.042 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-27T08:39:39.052 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.RWOrderedRead (3011 ms) 2024-03-27T08:39:39.052 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.ShortRead 2024-03-27T08:39:42.063 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.ShortRead (3010 ms) 2024-03-27T08:39:42.063 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.Exec 2024-03-27T08:39:45.076 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.Exec (3014 ms) 2024-03-27T08:39:45.077 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.Stat 2024-03-27T08:39:48.086 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.Stat (3009 ms) 2024-03-27T08:39:48.086 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.Omap 2024-03-27T08:39:51.101 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.Omap (3015 ms) 2024-03-27T08:39:51.101 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.OmapNuls 2024-03-27T08:39:54.115 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.OmapNuls (3014 ms) 2024-03-27T08:39:54.115 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.GetXattrs 2024-03-27T08:39:56.971 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.GetXattrs (2856 ms) 2024-03-27T08:39:56.971 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosReadOps.CmpExt 2024-03-27T08:39:59.979 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosReadOps.CmpExt (3009 ms) 2024-03-27T08:39:59.979 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 14 tests from NeoRadosReadOps (41989 ms total) 2024-03-27T08:39:59.980 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:39:59.980 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:39:59.980 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 14 tests from 1 test suite ran. (41989 ms total) 2024-03-27T08:39:59.980 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 14 tests. 2024-03-27T08:39:59.983 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:39:59.983 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:39:59.983 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:39:59.983 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' snapshots = ec_io ']' 2024-03-27T08:39:59.984 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' snapshots = ec_list ']' 2024-03-27T08:39:59.984 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_snapshots 2024-03-27T08:40:00.002 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:40:00.003 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 11 tests from 2 test suites. 2024-03-27T08:40:00.003 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:40:00.003 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from NeoRadosSnapshots 2024-03-27T08:40:00.003 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSnapshots.SnapList 2024-03-27T08:40:05.010 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSnapshots.SnapList (5008 ms) 2024-03-27T08:40:05.010 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-27T08:40:10.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSnapshots.SnapRemove (5018 ms) 2024-03-27T08:40:10.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSnapshots.Rollback 2024-03-27T08:40:14.045 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSnapshots.Rollback (4018 ms) 2024-03-27T08:40:14.046 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-27T08:40:18.978 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSnapshots.SnapGetName (4932 ms) 2024-03-27T08:40:18.978 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-27T08:40:26.005 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSnapshots.SnapCreateRemove (7028 ms) 2024-03-27T08:40:26.005 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 5 tests from NeoRadosSnapshots (26004 ms total) 2024-03-27T08:40:26.006 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:40:26.006 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-27T08:40:26.006 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-27T08:40:31.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSelfManagedSnaps.Snap (5022 ms) 2024-03-27T08:40:31.028 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-27T08:40:32.037 INFO:tasks.workunit.client.0.smithi104.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:40:33.048 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSelfManagedSnaps.Rollback (2020 ms) 2024-03-27T08:40:33.048 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-27T08:40:34.053 INFO:tasks.workunit.client.0.smithi104.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:40:35.064 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (2016 ms) 2024-03-27T08:40:35.064 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-27T08:40:41.077 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6013 ms) 2024-03-27T08:40:41.077 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-27T08:40:45.101 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4024 ms) 2024-03-27T08:40:45.101 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-27T08:40:48.109 INFO:tasks.workunit.client.0.smithi104.stdout:Deleting snap 3 in pool ReusePurgedSnapsmithi104-98981-11. 2024-03-27T08:40:49.119 INFO:tasks.workunit.client.0.smithi104.stdout:Waiting for snaps to purge. 2024-03-27T08:41:04.995 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19894 ms) 2024-03-27T08:41:04.995 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps (38989 ms total) 2024-03-27T08:41:04.996 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:41:04.996 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:41:04.996 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 11 tests from 2 test suites ran. (64993 ms total) 2024-03-27T08:41:04.996 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 11 tests. 2024-03-27T08:41:04.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:41:04.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:41:04.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:41:04.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' watch_notify = ec_io ']' 2024-03-27T08:41:04.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' watch_notify = ec_list ']' 2024-03-27T08:41:04.999 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_watch_notify 2024-03-27T08:41:05.020 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:41:05.020 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 2 tests from 1 test suite. 2024-03-27T08:41:05.020 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:41:05.020 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from NeoRadosWatchNotify 2024-03-27T08:41:05.020 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-27T08:41:07.487 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 28994896 notify_id 2800318676999 notifier_gid 15972 2024-03-27T08:41:08.006 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWatchNotify.WatchNotify (2987 ms) 2024-03-27T08:41:08.007 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-27T08:41:10.518 INFO:tasks.workunit.client.0.smithi104.stdout:Trying... 2024-03-27T08:41:10.528 INFO:tasks.workunit.client.0.smithi104.stdout:handle_notify cookie 41694512 notify_id 2813203578884 notifier_gid 15975 2024-03-27T08:41:10.529 INFO:tasks.workunit.client.0.smithi104.stdout:Waiting for 3.000000000s 2024-03-27T08:41:11.521 INFO:tasks.workunit.client.0.smithi104.stdout:Timed out. 2024-03-27T08:41:11.534 INFO:tasks.workunit.client.0.smithi104.stdout:Flushing... 2024-03-27T08:41:11.535 INFO:tasks.workunit.client.0.smithi104.stdout:Flushed... 2024-03-27T08:41:15.467 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7460 ms) 2024-03-27T08:41:15.467 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 2 tests from NeoRadosWatchNotify (10447 ms total) 2024-03-27T08:41:15.467 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:41:15.468 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:41:15.468 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 2 tests from 1 test suite ran. (10447 ms total) 2024-03-27T08:41:15.468 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 2 tests. 2024-03-27T08:41:15.470 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:41:15.470 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:41:15.470 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:41:15.470 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' write_operations = ec_io ']' 2024-03-27T08:41:15.470 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' write_operations = ec_list ']' 2024-03-27T08:41:15.470 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph_test_neorados_write_operations 2024-03-27T08:41:15.497 INFO:tasks.workunit.client.0.smithi104.stdout:Running main() from gmock_main.cc 2024-03-27T08:41:15.497 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] Running 7 tests from 1 test suite. 2024-03-27T08:41:15.497 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment set-up. 2024-03-27T08:41:15.498 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from NeoRadosWriteOps 2024-03-27T08:41:15.498 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.AssertExists 2024-03-27T08:41:18.482 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.AssertExists (2985 ms) 2024-03-27T08:41:18.482 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-27T08:41:21.498 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.AssertVersion (3016 ms) 2024-03-27T08:41:21.498 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.Xattrs 2024-03-27T08:41:24.002 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.Xattrs (2504 ms) 2024-03-27T08:41:24.002 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.Write 2024-03-27T08:41:27.015 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.Write (3013 ms) 2024-03-27T08:41:27.015 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.Exec 2024-03-27T08:41:30.024 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.Exec (3009 ms) 2024-03-27T08:41:30.024 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.WriteSame 2024-03-27T08:41:33.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.WriteSame (3009 ms) 2024-03-27T08:41:33.033 INFO:tasks.workunit.client.0.smithi104.stdout:[ RUN ] NeoRadosWriteOps.CmpExt 2024-03-27T08:41:43.001 INFO:tasks.workunit.client.0.smithi104.stdout:[ OK ] NeoRadosWriteOps.CmpExt (9968 ms) 2024-03-27T08:41:43.001 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] 7 tests from NeoRadosWriteOps (27504 ms total) 2024-03-27T08:41:43.001 INFO:tasks.workunit.client.0.smithi104.stdout: 2024-03-27T08:41:43.001 INFO:tasks.workunit.client.0.smithi104.stdout:[----------] Global test environment tear-down 2024-03-27T08:41:43.001 INFO:tasks.workunit.client.0.smithi104.stdout:[==========] 7 tests from 1 test suite ran. (27504 ms total) 2024-03-27T08:41:43.001 INFO:tasks.workunit.client.0.smithi104.stdout:[ PASSED ] 7 tests. 2024-03-27T08:41:43.003 INFO:tasks.workunit.client.0.smithi104.stderr:+ ret=0 2024-03-27T08:41:43.003 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:41:43.003 INFO:tasks.workunit.client.0.smithi104.stderr:+ exit 0 2024-03-27T08:41:43.003 INFO:tasks.workunit.client.0.smithi104.stderr:+ cleanup 2024-03-27T08:41:43.003 INFO:tasks.workunit.client.0.smithi104.stderr:+ pkill -P 33402 2024-03-27T08:41:43.092 INFO:tasks.workunit.client.0.smithi104.stderr:+ true 2024-03-27T08:41:43.093 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-27T08:41:43.093 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-27T08:41:43.163 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-27T08:41:43.164 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-27T08:41:43.164 DEBUG:teuthology.orchestra.run.smithi104:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=eae914f097640d21427554e7f6aa6f1e9c4a7748 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-27T08:41:43.222 INFO:tasks.workunit.client.0.smithi104.stderr:++ uuidgen 2024-03-27T08:41:43.234 INFO:tasks.workunit.client.0.smithi104.stderr:+ p=20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:41:43.234 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool create 20a90cab-b0dc-43af-9f0a-bd39527c62c7 12 2024-03-27T08:41:43.334 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.327+0000 7f1889b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:43.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.344+0000 7f1889b9d640 1 Processor -- start 2024-03-27T08:41:43.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.344+0000 7f1889b9d640 1 -- start start 2024-03-27T08:41:43.338 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.346+0000 7f1889b9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f188409f810 0x7f188409fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:43.338 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.346+0000 7f1889b9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188406dcd0 con 0x7f18841347d0 2024-03-27T08:41:43.338 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.346+0000 7f1889b9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f18840a0160 con 0x7f188409f810 2024-03-27T08:41:43.338 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.346+0000 7f1882ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f18841347d0 0x7f1884134be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53640/0 (socket says 172.21.15.104:53640) 2024-03-27T08:41:43.339 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.346+0000 7f18837fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f188409f810 0x7f188409fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:43.339 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.346+0000 7f1882ffd640 1 -- 172.21.15.104:0/68181150 learned_addr learned my addr 172.21.15.104:0/68181150 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.347+0000 7f18827fc640 1 -- 172.21.15.104:0/68181150 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236831838 0 0) 0x7f188406dcd0 con 0x7f18841347d0 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.347+0000 7f18827fc640 1 -- 172.21.15.104:0/68181150 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f186c003580 con 0x7f18841347d0 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.347+0000 7f18837fe640 1 -- 172.21.15.104:0/68181150 >> v1:172.21.15.104:6789/0 conn(0x7f18841347d0 legacy=0x7f1884134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.347+0000 7f18837fe640 1 -- 172.21.15.104:0/68181150 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188409ee10 con 0x7f188409f810 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.348+0000 7f18837fe640 1 --2- 172.21.15.104:0/68181150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f188409f810 0x7f188409fc20 crc :-1 s=READY pgs=12839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6bed06af7f3609 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.348+0000 7f18827fc640 1 -- 172.21.15.104:0/68181150 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1878009d90 con 0x7f188409f810 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.348+0000 7f18827fc640 1 -- 172.21.15.104:0/68181150 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f187800e230 con 0x7f188409f810 2024-03-27T08:41:43.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.348+0000 7f18827fc640 1 -- 172.21.15.104:0/68181150 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f187800e4d0 con 0x7f188409f810 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.349+0000 7f1889b9d640 1 -- 172.21.15.104:0/68181150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f188409f810 msgr2=0x7f188409fc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.349+0000 7f1889b9d640 1 --2- 172.21.15.104:0/68181150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f188409f810 0x7f188409fc20 crc :-1 s=READY pgs=12839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.349+0000 7f1889b9d640 1 -- 172.21.15.104:0/68181150 shutdown_connections 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.349+0000 7f1889b9d640 1 --2- 172.21.15.104:0/68181150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f188409f810 0x7f188409fc20 crc :-1 s=CLOSED pgs=12839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.349+0000 7f1889b9d640 1 -- 172.21.15.104:0/68181150 >> 172.21.15.104:0/68181150 conn(0x7f18840a4d30 msgr2=0x7f18840a5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.350+0000 7f1889b9d640 1 -- 172.21.15.104:0/68181150 shutdown_connections 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.350+0000 7f1889b9d640 1 -- 172.21.15.104:0/68181150 wait complete. 2024-03-27T08:41:43.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.350+0000 7f1889b9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:43.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.352+0000 7f1889b9d640 1 Processor -- start 2024-03-27T08:41:43.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.353+0000 7f1889b9d640 1 -- start start 2024-03-27T08:41:43.346 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.354+0000 7f1889b9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 0x7f188411f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:43.346 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.354+0000 7f1889b9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1884121740 con 0x7f18841347d0 2024-03-27T08:41:43.346 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.354+0000 7f18837fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 0x7f188411f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:43.346 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.354+0000 7f18837fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 0x7f188411f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34042/0 (socket says 172.21.15.104:34042) 2024-03-27T08:41:43.347 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.354+0000 7f18837fe640 1 -- 172.21.15.104:0/552600525 learned_addr learned my addr 172.21.15.104:0/552600525 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:43.347 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f18837fe640 1 -- 172.21.15.104:0/552600525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1884123230 con 0x7f18841347d0 2024-03-27T08:41:43.347 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f18837fe640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 0x7f188411f250 crc :-1 s=READY pgs=12840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:43.347 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1878013dd0 con 0x7f18841347d0 2024-03-27T08:41:43.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1884120d40 con 0x7f18841347d0 2024-03-27T08:41:43.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1878014410 con 0x7f18841347d0 2024-03-27T08:41:43.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f18780146b0 con 0x7f18841347d0 2024-03-27T08:41:43.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.355+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1884121ca0 con 0x7f18841347d0 2024-03-27T08:41:43.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.356+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18840693c0 con 0x7f18841347d0 2024-03-27T08:41:43.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.357+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f18780148b0 con 0x7f18841347d0 2024-03-27T08:41:43.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.357+0000 7f186bfff640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1870038750 0x7f187003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:43.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.357+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(681..681 src has 397..681) ==== 3403+0+0 (crc 0 0 0) 0x7f1878019040 con 0x7f18841347d0 2024-03-27T08:41:43.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.357+0000 7f1882ffd640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1870038750 0x7f187003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:43.350 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.358+0000 7f1882ffd640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1870038750 0x7f187003ac60 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:43.355 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.363+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f1878019710 con 0x7f18841347d0 2024-03-27T08:41:43.686 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:43.694+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pg_num": 12} v 0) -- 0x7f188409f3c0 con 0x7f18841347d0 2024-03-27T08:41:44.001 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.008+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pg_num": 12}]=0 pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' created v682) ==== 176+0+0 (crc 0 0 0) 0x7f1878019710 con 0x7f18841347d0 2024-03-27T08:41:44.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.151+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pg_num": 12} v 0) -- 0x7f18840008d0 con 0x7f18841347d0 2024-03-27T08:41:44.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.151+0000 7f186bfff640 1 -- 172.21.15.104:0/552600525 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pg_num": 12}]=0 pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' already exists v682) ==== 183+0+0 (crc 0 0 0) 0x7f18780198f0 con 0x7f18841347d0 2024-03-27T08:41:44.147 INFO:tasks.workunit.client.0.smithi104.stderr:pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' already exists 2024-03-27T08:41:44.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1870038750 msgr2=0x7f187003ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:44.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1870038750 0x7f187003ac60 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:44.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 msgr2=0x7f188411f250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:44.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 0x7f188411f250 crc :-1 s=READY pgs=12840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:44.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 shutdown_connections 2024-03-27T08:41:44.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1870038750 0x7f187003ac60 unknown :-1 s=CLOSED pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:44.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 --2- 172.21.15.104:0/552600525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18841347d0 0x7f188411f250 unknown :-1 s=CLOSED pgs=12840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:44.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 >> 172.21.15.104:0/552600525 conn(0x7f18840a4d30 msgr2=0x7f18841bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:44.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 shutdown_connections 2024-03-27T08:41:44.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.155+0000 7f1889b9d640 1 -- 172.21.15.104:0/552600525 wait complete. 2024-03-27T08:41:44.161 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_objects 10 2024-03-27T08:41:44.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.270+0000 7fb0210ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:44.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.285+0000 7fb0210ed640 1 Processor -- start 2024-03-27T08:41:44.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.285+0000 7fb0210ed640 1 -- start start 2024-03-27T08:41:44.279 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb0210ed640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb01c137b10 0x7fb01c139f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:44.279 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb0210ed640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb01c06eb50 con 0x7fb01c09f6e0 2024-03-27T08:41:44.279 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb0210ed640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fb01c09f2f0 con 0x7fb01c137b10 2024-03-27T08:41:44.279 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb01a59c640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb01c137b10 0x7fb01c139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:44.279 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb01ad9d640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb01c09f6e0 0x7fb01c09faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:53654/0 (socket says 172.21.15.104:53654) 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb01ad9d640 1 -- 172.21.15.104:0/3000266351 learned_addr learned my addr 172.21.15.104:0/3000266351 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb01a59c640 1 -- 172.21.15.104:0/3000266351 >> v1:172.21.15.104:6789/0 conn(0x7fb01c09f6e0 legacy=0x7fb01c09faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb01a59c640 1 -- 172.21.15.104:0/3000266351 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c0a0200 con 0x7fb01c137b10 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb01a59c640 1 --2- 172.21.15.104:0/3000266351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c137b10 0x7fb01c139f30 crc :-1 s=READY pgs=12842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344bdda47059f7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb019d9b640 1 -- 172.21.15.104:0/3000266351 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb010009d90 con 0x7fb01c137b10 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb019d9b640 1 -- 172.21.15.104:0/3000266351 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb01000e230 con 0x7fb01c137b10 2024-03-27T08:41:44.280 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.287+0000 7fb019d9b640 1 -- 172.21.15.104:0/3000266351 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb01000e4d0 con 0x7fb01c137b10 2024-03-27T08:41:44.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 -- 172.21.15.104:0/3000266351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c137b10 msgr2=0x7fb01c139f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:44.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 --2- 172.21.15.104:0/3000266351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c137b10 0x7fb01c139f30 crc :-1 s=READY pgs=12842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:44.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 -- 172.21.15.104:0/3000266351 shutdown_connections 2024-03-27T08:41:44.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 --2- 172.21.15.104:0/3000266351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c137b10 0x7fb01c139f30 unknown :-1 s=CLOSED pgs=12842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:44.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 -- 172.21.15.104:0/3000266351 >> 172.21.15.104:0/3000266351 conn(0x7fb01c0a1860 msgr2=0x7fb01c0a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:44.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 -- 172.21.15.104:0/3000266351 shutdown_connections 2024-03-27T08:41:44.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 1 -- 172.21.15.104:0/3000266351 wait complete. 2024-03-27T08:41:44.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.289+0000 7fb0210ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:44.283 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.291+0000 7fb0210ed640 1 Processor -- start 2024-03-27T08:41:44.283 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.291+0000 7fb0210ed640 1 -- start start 2024-03-27T08:41:44.284 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.292+0000 7fb0210ed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 0x7fb01c11ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:44.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.292+0000 7fb0210ed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c122ee0 con 0x7fb01c09f6e0 2024-03-27T08:41:44.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.292+0000 7fb01ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 0x7fb01c11ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:44.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.292+0000 7fb01ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 0x7fb01c11ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34064/0 (socket says 172.21.15.104:34064) 2024-03-27T08:41:44.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.292+0000 7fb01ad9d640 1 -- 172.21.15.104:0/3235427017 learned_addr learned my addr 172.21.15.104:0/3235427017 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:44.286 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb01ad9d640 1 -- 172.21.15.104:0/3235427017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c131f40 con 0x7fb01c09f6e0 2024-03-27T08:41:44.286 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb01ad9d640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 0x7fb01c11ed50 crc :-1 s=READY pgs=12843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:44.286 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb00c005a60 con 0x7fb01c09f6e0 2024-03-27T08:41:44.286 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb00c005bc0 con 0x7fb01c09f6e0 2024-03-27T08:41:44.286 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c121260 con 0x7fb01c09f6e0 2024-03-27T08:41:44.287 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb00c005e60 con 0x7fb01c09f6e0 2024-03-27T08:41:44.287 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.293+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1223c0 con 0x7fb01c09f6e0 2024-03-27T08:41:44.287 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.294+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fb00c00c780 con 0x7fb01c09f6e0 2024-03-27T08:41:44.288 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.295+0000 7fb0037fe640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb008038750 0x7fb00803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:44.288 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.295+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(682..682 src has 397..682) ==== 3778+0+0 (crc 0 0 0) 0x7fb00c048440 con 0x7fb01c09f6e0 2024-03-27T08:41:44.288 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.295+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c122c90 con 0x7fb01c09f6e0 2024-03-27T08:41:44.288 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.295+0000 7fb01a59c640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb008038750 0x7fb00803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:44.288 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.296+0000 7fb01a59c640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb008038750 0x7fb00803ac60 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:44.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.305+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fb01c122c90 con 0x7fb01c09f6e0 2024-03-27T08:41:44.479 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:44.486+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "10"} v 0) -- 0x7fb01c122c90 con 0x7fb01c09f6e0 2024-03-27T08:41:45.007 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:45.015+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v683) ==== 223+0+0 (crc 0 0 0) 0x7fb01c122c90 con 0x7fb01c09f6e0 2024-03-27T08:41:45.210 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:45.217+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "10"} v 0) -- 0x7fb01c122c90 con 0x7fb01c09f6e0 2024-03-27T08:41:46.017 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.025+0000 7fb0037fe640 1 -- 172.21.15.104:0/3235427017 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v684) ==== 223+0+0 (crc 0 0 0) 0x7fb01c122c90 con 0x7fb01c09f6e0 2024-03-27T08:41:46.017 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_objects = 10 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:41:46.019 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb008038750 msgr2=0x7fb00803ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb008038750 0x7fb00803ac60 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 msgr2=0x7fb01c11ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 0x7fb01c11ed50 crc :-1 s=READY pgs=12843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 shutdown_connections 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb008038750 0x7fb00803ac60 unknown :-1 s=CLOSED pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 --2- 172.21.15.104:0/3235427017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb01c09f6e0 0x7fb01c11ed50 unknown :-1 s=CLOSED pgs=12843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 >> 172.21.15.104:0/3235427017 conn(0x7fb01c0a1860 msgr2=0x7fb01c134610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 shutdown_connections 2024-03-27T08:41:46.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.027+0000 7fb0210ed640 1 -- 172.21.15.104:0/3235427017 wait complete. 2024-03-27T08:41:46.029 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool application enable 20a90cab-b0dc-43af-9f0a-bd39527c62c7 rados 2024-03-27T08:41:46.121 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.121+0000 7fa641a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:46.122 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.130+0000 7fa641a6d640 1 Processor -- start 2024-03-27T08:41:46.122 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.130+0000 7fa641a6d640 1 -- start start 2024-03-27T08:41:46.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa641a6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa63c12f670 0x7fa63c12fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:46.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa641a6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa63c075450 con 0x7fa63c132690 2024-03-27T08:41:46.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa641a6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fa63c072630 con 0x7fa63c12f670 2024-03-27T08:41:46.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa63b7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa63c12f670 0x7fa63c12fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:46.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa63b7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa63c12f670 0x7fa63c12fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46942/0 (socket says 172.21.15.104:46942) 2024-03-27T08:41:46.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa63affd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa63c132690 0x7fa63c132aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:55952/0 (socket says 172.21.15.104:55952) 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.131+0000 7fa63b7fe640 1 -- 172.21.15.104:0/120808316 learned_addr learned my addr 172.21.15.104:0/120808316 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63b7fe640 1 -- 172.21.15.104:0/120808316 >> v1:172.21.15.104:6789/0 conn(0x7fa63c132690 legacy=0x7fa63c132aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63b7fe640 1 -- 172.21.15.104:0/120808316 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c073060 con 0x7fa63c12f670 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63a7fc640 1 -- 172.21.15.104:0/120808316 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685459478 0 0) 0x7fa63c075450 con 0x7fa63c132690 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63b7fe640 1 --2- 172.21.15.104:0/120808316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c12fa80 crc :-1 s=READY pgs=12845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e562f93ff5043b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63a7fc640 1 -- 172.21.15.104:0/120808316 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa630009e50 con 0x7fa63c12f670 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63a7fc640 1 -- 172.21.15.104:0/120808316 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa63000e230 con 0x7fa63c12f670 2024-03-27T08:41:46.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.132+0000 7fa63a7fc640 1 -- 172.21.15.104:0/120808316 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa63000e4d0 con 0x7fa63c12f670 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 -- 172.21.15.104:0/120808316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 msgr2=0x7fa63c12fa80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 --2- 172.21.15.104:0/120808316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c12fa80 crc :-1 s=READY pgs=12845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 -- 172.21.15.104:0/120808316 shutdown_connections 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 --2- 172.21.15.104:0/120808316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c12fa80 unknown :-1 s=CLOSED pgs=12845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 -- 172.21.15.104:0/120808316 >> 172.21.15.104:0/120808316 conn(0x7fa63c0a0b00 msgr2=0x7fa63c0a0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 -- 172.21.15.104:0/120808316 shutdown_connections 2024-03-27T08:41:46.126 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 1 -- 172.21.15.104:0/120808316 wait complete. 2024-03-27T08:41:46.127 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.133+0000 7fa641a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:46.127 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.134+0000 7fa641a6d640 1 Processor -- start 2024-03-27T08:41:46.127 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.134+0000 7fa641a6d640 1 -- start start 2024-03-27T08:41:46.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.135+0000 7fa641a6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c127cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:46.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.135+0000 7fa641a6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c09c450 con 0x7fa63c12f670 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.135+0000 7fa63b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c127cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.135+0000 7fa63b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c127cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46950/0 (socket says 172.21.15.104:46950) 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.135+0000 7fa63b7fe640 1 -- 172.21.15.104:0/3773113266 learned_addr learned my addr 172.21.15.104:0/3773113266 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa63b7fe640 1 -- 172.21.15.104:0/3773113266 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c09c640 con 0x7fa63c12f670 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa63b7fe640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c127cc0 crc :-1 s=READY pgs=12846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa63000ed90 con 0x7fa63c12f670 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa630015420 con 0x7fa63c12f670 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa6300156c0 con 0x7fa63c12f670 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c09cf80 con 0x7fa63c12f670 2024-03-27T08:41:46.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.136+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c12c6c0 con 0x7fa63c12f670 2024-03-27T08:41:46.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.137+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fa6300158c0 con 0x7fa63c12f670 2024-03-27T08:41:46.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.137+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c12aeb0 con 0x7fa63c12f670 2024-03-27T08:41:46.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.137+0000 7fa623fff640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa628038750 0x7fa62803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:46.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.137+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(684..684 src has 397..684) ==== 3778+0+0 (crc 0 0 0) 0x7fa6300499a0 con 0x7fa63c12f670 2024-03-27T08:41:46.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.137+0000 7fa63affd640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa628038750 0x7fa62803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:46.131 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.138+0000 7fa63affd640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa628038750 0x7fa62803ac60 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:46.134 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.142+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fa630010a00 con 0x7fa63c12f670 2024-03-27T08:41:46.311 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:46.318+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "app": "rados"} v 0) -- 0x7fa63c0693c0 con 0x7fa63c12f670 2024-03-27T08:41:47.023 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:47.031+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "app": "rados"}]=0 enabled application 'rados' on pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' v685) ==== 213+0+0 (crc 0 0 0) 0x7fa630010e60 con 0x7fa63c12f670 2024-03-27T08:41:47.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:47.143+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "app": "rados"} v 0) -- 0x7fa63c0008d0 con 0x7fa63c12f670 2024-03-27T08:41:48.032 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.040+0000 7fa623fff640 1 -- 172.21.15.104:0/3773113266 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "app": "rados"}]=0 enabled application 'rados' on pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' v686) ==== 213+0+0 (crc 0 0 0) 0x7fa630010820 con 0x7fa63c12f670 2024-03-27T08:41:48.032 INFO:tasks.workunit.client.0.smithi104.stderr:enabled application 'rados' on pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' 2024-03-27T08:41:48.035 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa628038750 msgr2=0x7fa62803ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa628038750 0x7fa62803ac60 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 msgr2=0x7fa63c127cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c127cc0 crc :-1 s=READY pgs=12846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 shutdown_connections 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa628038750 0x7fa62803ac60 unknown :-1 s=CLOSED pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 --2- 172.21.15.104:0/3773113266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63c12f670 0x7fa63c127cc0 unknown :-1 s=CLOSED pgs=12846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 >> 172.21.15.104:0/3773113266 conn(0x7fa63c0a0b00 msgr2=0x7fa63c1ba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 shutdown_connections 2024-03-27T08:41:48.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.043+0000 7fa641a6d640 1 -- 172.21.15.104:0/3773113266 wait complete. 2024-03-27T08:41:48.050 INFO:tasks.workunit.client.0.smithi104.stderr:++ seq 1 10 2024-03-27T08:41:48.058 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.058 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj1 /etc/passwd 2024-03-27T08:41:48.099 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.106+0000 7fd43dc31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.112+0000 7fd43dc31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.105 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.112+0000 7fd43dc31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.105 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.113+0000 7fd43dc31ec0 1 Processor -- start 2024-03-27T08:41:48.106 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.113+0000 7fd43dc31ec0 1 -- start start 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.115+0000 7fd43dc31ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x1113c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.115+0000 7fd43dc31ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xf5c0b0 con 0xff6b20 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.115+0000 7fd43559a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x1113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.115+0000 7fd43559a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x1113c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46978/0 (socket says 172.21.15.104:46978) 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.115+0000 7fd43559a640 1 -- 172.21.15.104:0/3833507811 learned_addr learned my addr 172.21.15.104:0/3833507811 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.115+0000 7fd43559a640 1 -- 172.21.15.104:0/3833507811 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xefc1e0 con 0xff6b20 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.116+0000 7fd43559a640 1 --2- 172.21.15.104:0/3833507811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x1113c10 crc :-1 s=READY pgs=12849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ee0577819b1cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.116+0000 7fd4277fe640 1 -- 172.21.15.104:0/3833507811 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd42c00b500 con 0xff6b20 2024-03-27T08:41:48.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.116+0000 7fd4277fe640 1 -- 172.21.15.104:0/3833507811 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd42c00b660 con 0xff6b20 2024-03-27T08:41:48.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.116+0000 7fd4277fe640 1 -- 172.21.15.104:0/3833507811 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd42c00b900 con 0xff6b20 2024-03-27T08:41:48.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/3833507811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 msgr2=0x1113c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/3833507811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x1113c10 crc :-1 s=READY pgs=12849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/3833507811 shutdown_connections 2024-03-27T08:41:48.110 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/3833507811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x1113c10 unknown :-1 s=CLOSED pgs=12849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.110 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/3833507811 >> 172.21.15.104:0/3833507811 conn(0xf10ea0 msgr2=0xff0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.110 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/3833507811 shutdown_connections 2024-03-27T08:41:48.110 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.117+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/3833507811 wait complete. 2024-03-27T08:41:48.110 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.118+0000 7fd43dc31ec0 1 Processor -- start 2024-03-27T08:41:48.110 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.118+0000 7fd43dc31ec0 1 -- start start 2024-03-27T08:41:48.111 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.119+0000 7fd43dc31ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x108b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.112 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.119+0000 7fd43dc31ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xf59b30 con 0xff6b20 2024-03-27T08:41:48.112 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.119+0000 7fd43559a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x108b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.112 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.119+0000 7fd43559a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x108b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46984/0 (socket says 172.21.15.104:46984) 2024-03-27T08:41:48.112 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.119+0000 7fd43559a640 1 -- 172.21.15.104:0/1386007975 learned_addr learned my addr 172.21.15.104:0/1386007975 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.112 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.119+0000 7fd43559a640 1 -- 172.21.15.104:0/1386007975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xeeee30 con 0xff6b20 2024-03-27T08:41:48.112 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.120+0000 7fd43559a640 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x108b9c0 crc :-1 s=READY pgs=12850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.120+0000 7fd4267fc640 1 -- 172.21.15.104:0/1386007975 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd42c0111e0 con 0xff6b20 2024-03-27T08:41:48.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.120+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xeee890 con 0xff6b20 2024-03-27T08:41:48.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.120+0000 7fd4267fc640 1 -- 172.21.15.104:0/1386007975 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd42c011a00 con 0xff6b20 2024-03-27T08:41:48.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.120+0000 7fd4267fc640 1 -- 172.21.15.104:0/1386007975 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd42c011b70 con 0xff6b20 2024-03-27T08:41:48.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.120+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x108cb20 con 0xff6b20 2024-03-27T08:41:48.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.121+0000 7fd4267fc640 1 -- 172.21.15.104:0/1386007975 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fd42c00f450 con 0xff6b20 2024-03-27T08:41:48.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.121+0000 7fd4267fc640 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd414036fe0 0x7fd4140394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.121+0000 7fd4267fc640 1 -- 172.21.15.104:0/1386007975 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7fd42c0489c0 con 0xff6b20 2024-03-27T08:41:48.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.122+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> v2:172.21.15.177:6807/929709834 conn(0xff7640 0x1078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.122+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 200.7 200:ff7b1f36:::obj1:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x107bb40 con 0xff7640 2024-03-27T08:41:48.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.122+0000 7fd434d99640 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd414036fe0 0x7fd4140394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.122+0000 7fd434d99640 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd414036fe0 0x7fd4140394f0 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.123+0000 7fd435d9b640 1 --2- 172.21.15.104:0/1386007975 >> v2:172.21.15.177:6807/929709834 conn(0xff7640 0x1078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:48.118 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.126+0000 7fd435d9b640 1 --2- 172.21.15.104:0/1386007975 >> v2:172.21.15.177:6807/929709834 conn(0xff7640 0x1078580 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd435d9b640 1 -- 172.21.15.104:0/1386007975 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj1 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fd42800a430 con 0xff7640 2024-03-27T08:41:48.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 >> v2:172.21.15.177:6807/929709834 conn(0xff7640 msgr2=0x1078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> v2:172.21.15.177:6807/929709834 conn(0xff7640 0x1078580 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd414036fe0 msgr2=0x7fd4140394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd414036fe0 0x7fd4140394f0 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 msgr2=0x108b9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x108b9c0 crc :-1 s=READY pgs=12850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 shutdown_connections 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> v2:172.21.15.177:6807/929709834 conn(0xff7640 0x1078580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd414036fe0 0x7fd4140394f0 unknown :-1 s=CLOSED pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 --2- 172.21.15.104:0/1386007975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xff6b20 0x108b9c0 unknown :-1 s=CLOSED pgs=12850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 >> 172.21.15.104:0/1386007975 conn(0xf10ea0 msgr2=0xff0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 shutdown_connections 2024-03-27T08:41:48.484 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.490+0000 7fd43dc31ec0 1 -- 172.21.15.104:0/1386007975 wait complete. 2024-03-27T08:41:48.485 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.485 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj2 /etc/passwd 2024-03-27T08:41:48.518 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.526+0000 7f66c8e2cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.523 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.530+0000 7f66c8e2cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.523 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.531+0000 7f66c8e2cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.523 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.531+0000 7f66c8e2cec0 1 Processor -- start 2024-03-27T08:41:48.523 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.531+0000 7f66c8e2cec0 1 -- start start 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.532+0000 7f66c8e2cec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc80080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.532+0000 7f66c8e2cec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xacd0b0 con 0xb688c0 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.532+0000 7f66c0f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc80080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.532+0000 7f66c0f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc80080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46996/0 (socket says 172.21.15.104:46996) 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.533+0000 7f66c0f9b640 1 -- 172.21.15.104:0/529698138 learned_addr learned my addr 172.21.15.104:0/529698138 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.533+0000 7f66c0f9b640 1 -- 172.21.15.104:0/529698138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xa6d1e0 con 0xb688c0 2024-03-27T08:41:48.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.533+0000 7f66c0f9b640 1 --2- 172.21.15.104:0/529698138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc80080 crc :-1 s=READY pgs=12853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd8772f39929435 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.533+0000 7f66ba7fc640 1 -- 172.21.15.104:0/529698138 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f66ac033040 con 0xb688c0 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.533+0000 7f66ba7fc640 1 -- 172.21.15.104:0/529698138 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f66ac034040 con 0xb688c0 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.533+0000 7f66ba7fc640 1 -- 172.21.15.104:0/529698138 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f66ac02eb30 con 0xb688c0 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/529698138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 msgr2=0xc80080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/529698138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc80080 crc :-1 s=READY pgs=12853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/529698138 shutdown_connections 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/529698138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc80080 unknown :-1 s=CLOSED pgs=12853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/529698138 >> 172.21.15.104:0/529698138 conn(0xa81ea0 msgr2=0xb63a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/529698138 shutdown_connections 2024-03-27T08:41:48.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.534+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/529698138 wait complete. 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.535+0000 7f66c8e2cec0 1 Processor -- start 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.535+0000 7f66c8e2cec0 1 -- start start 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c8e2cec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc8fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c8e2cec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xacab30 con 0xb688c0 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c0f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc8fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c0f9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc8fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:46998/0 (socket says 172.21.15.104:46998) 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c0f9b640 1 -- 172.21.15.104:0/2246972836 learned_addr learned my addr 172.21.15.104:0/2246972836 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c0f9b640 1 -- 172.21.15.104:0/2246972836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xa5fe30 con 0xb688c0 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c0f9b640 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc8fe20 crc :-1 s=READY pgs=12854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66b97fa640 1 -- 172.21.15.104:0/2246972836 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f66ac033040 con 0xb688c0 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66b97fa640 1 -- 172.21.15.104:0/2246972836 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f66ac02d7f0 con 0xb688c0 2024-03-27T08:41:48.529 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xa5f890 con 0xb688c0 2024-03-27T08:41:48.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66b97fa640 1 -- 172.21.15.104:0/2246972836 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f66ac02da90 con 0xb688c0 2024-03-27T08:41:48.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.536+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0xc7d8f0 con 0xb688c0 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.537+0000 7f66b97fa640 1 -- 172.21.15.104:0/2246972836 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f66ac02dc40 con 0xb688c0 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.537+0000 7f66b97fa640 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f66a8037030 0x7f66a8039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.538+0000 7f66b97fa640 1 -- 172.21.15.104:0/2246972836 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f66ac000e40 con 0xb688c0 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.538+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> v2:172.21.15.177:6807/929709834 conn(0xc696c0 0xc69ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.538+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 200.8 200:104778fc:::obj2:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0xc6ccb0 con 0xc696c0 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.538+0000 7f66bb7fe640 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f66a8037030 0x7f66a8039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.538+0000 7f66bb7fe640 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f66a8037030 0x7f66a8039540 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.538+0000 7f66c179c640 1 --2- 172.21.15.104:0/2246972836 >> v2:172.21.15.177:6807/929709834 conn(0xc696c0 0xc69ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:48.533 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.541+0000 7f66c179c640 1 --2- 172.21.15.104:0/2246972836 >> v2:172.21.15.177:6807/929709834 conn(0xc696c0 0xc69ad0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.545 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c179c640 1 -- 172.21.15.104:0/2246972836 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj2 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f66b400a430 con 0xc696c0 2024-03-27T08:41:48.545 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 >> v2:172.21.15.177:6807/929709834 conn(0xc696c0 msgr2=0xc69ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.545 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> v2:172.21.15.177:6807/929709834 conn(0xc696c0 0xc69ad0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f66a8037030 msgr2=0x7f66a8039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f66a8037030 0x7f66a8039540 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 msgr2=0xc8fe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc8fe20 crc :-1 s=READY pgs=12854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 shutdown_connections 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> v2:172.21.15.177:6807/929709834 conn(0xc696c0 0xc69ad0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f66a8037030 0x7f66a8039540 unknown :-1 s=CLOSED pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 --2- 172.21.15.104:0/2246972836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb688c0 0xc8fe20 unknown :-1 s=CLOSED pgs=12854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 >> 172.21.15.104:0/2246972836 conn(0xa81ea0 msgr2=0xb63a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 shutdown_connections 2024-03-27T08:41:48.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.552+0000 7f66c8e2cec0 1 -- 172.21.15.104:0/2246972836 wait complete. 2024-03-27T08:41:48.547 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.547 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj3 /etc/passwd 2024-03-27T08:41:48.581 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.589+0000 7f6013e29ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.586 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.594+0000 7f6013e29ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.586 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.594+0000 7f6013e29ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.587 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.595+0000 7f6013e29ec0 1 Processor -- start 2024-03-27T08:41:48.587 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.595+0000 7f6013e29ec0 1 -- start start 2024-03-27T08:41:48.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6013e29ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x176b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6013e29ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x16d70b0 con 0x176dc90 2024-03-27T08:41:48.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6006ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x176b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6006ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x176b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47014/0 (socket says 172.21.15.104:47014) 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6006ffd640 1 -- 172.21.15.104:0/2419089340 learned_addr learned my addr 172.21.15.104:0/2419089340 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6006ffd640 1 -- 172.21.15.104:0/2419089340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x16771e0 con 0x176dc90 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f6006ffd640 1 --2- 172.21.15.104:0/2419089340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x176b8c0 crc :-1 s=READY pgs=12857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b576f95ed60279ca server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f60057fa640 1 -- 172.21.15.104:0/2419089340 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ff80060b0 con 0x176dc90 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f60057fa640 1 -- 172.21.15.104:0/2419089340 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5ff8006210 con 0x176dc90 2024-03-27T08:41:48.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.596+0000 7f60057fa640 1 -- 172.21.15.104:0/2419089340 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ff8001b10 con 0x176dc90 2024-03-27T08:41:48.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 -- 172.21.15.104:0/2419089340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 msgr2=0x176b8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/2419089340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x176b8c0 crc :-1 s=READY pgs=12857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 -- 172.21.15.104:0/2419089340 shutdown_connections 2024-03-27T08:41:48.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/2419089340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x176b8c0 unknown :-1 s=CLOSED pgs=12857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 -- 172.21.15.104:0/2419089340 >> 172.21.15.104:0/2419089340 conn(0x168bea0 msgr2=0x1808040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 -- 172.21.15.104:0/2419089340 shutdown_connections 2024-03-27T08:41:48.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.597+0000 7f6013e29ec0 1 -- 172.21.15.104:0/2419089340 wait complete. 2024-03-27T08:41:48.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.598+0000 7f6013e29ec0 1 Processor -- start 2024-03-27T08:41:48.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.598+0000 7f6013e29ec0 1 -- start start 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6013e29ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x189ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6013e29ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x16d4b30 con 0x176dc90 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6006ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x189ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6006ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x189ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47020/0 (socket says 172.21.15.104:47020) 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6006ffd640 1 -- 172.21.15.104:0/1316873609 learned_addr learned my addr 172.21.15.104:0/1316873609 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6006ffd640 1 -- 172.21.15.104:0/1316873609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1669e30 con 0x176dc90 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6006ffd640 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x189ab10 crc :-1 s=READY pgs=12858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f5feffff640 1 -- 172.21.15.104:0/1316873609 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ff8004aa0 con 0x176dc90 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f5feffff640 1 -- 172.21.15.104:0/1316873609 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5ff8004c00 con 0x176dc90 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1669890 con 0x176dc90 2024-03-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f5feffff640 1 -- 172.21.15.104:0/1316873609 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ff8004ea0 con 0x176dc90 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.599+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1776b70 con 0x176dc90 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.600+0000 7f5feffff640 1 -- 172.21.15.104:0/1316873609 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f5ff8010440 con 0x176dc90 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.600+0000 7f5feffff640 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5ff4037030 0x7f5ff4039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.600+0000 7f5feffff640 1 -- 172.21.15.104:0/1316873609 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f5ff80497c0 con 0x176dc90 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.601+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> v2:172.21.15.104:6801/1224298329 conn(0x176f880 0x18032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.601+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 200.1 200:8dd16f86:::obj3:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x17ee050 con 0x176f880 2024-03-27T08:41:48.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.601+0000 7f60067fc640 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5ff4037030 0x7f5ff4039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.601+0000 7f60067fc640 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5ff4037030 0x7f5ff4039540 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.601+0000 7f60077fe640 1 --2- 172.21.15.104:0/1316873609 >> v2:172.21.15.104:6801/1224298329 conn(0x176f880 0x18032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:48.600 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.607+0000 7f60077fe640 1 --2- 172.21.15.104:0/1316873609 >> v2:172.21.15.104:6801/1224298329 conn(0x176f880 0x18032a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.780 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f60077fe640 1 -- 172.21.15.104:0/1316873609 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj3 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f600000c430 con 0x176f880 2024-03-27T08:41:48.780 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 >> v2:172.21.15.104:6801/1224298329 conn(0x176f880 msgr2=0x18032a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.780 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> v2:172.21.15.104:6801/1224298329 conn(0x176f880 0x18032a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.780 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5ff4037030 msgr2=0x7f5ff4039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5ff4037030 0x7f5ff4039540 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 msgr2=0x189ab10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x189ab10 crc :-1 s=READY pgs=12858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 shutdown_connections 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5ff4037030 0x7f5ff4039540 unknown :-1 s=CLOSED pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> v2:172.21.15.104:6801/1224298329 conn(0x176f880 0x18032a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 --2- 172.21.15.104:0/1316873609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x176dc90 0x189ab10 unknown :-1 s=CLOSED pgs=12858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 >> 172.21.15.104:0/1316873609 conn(0x168bea0 msgr2=0x1771010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 shutdown_connections 2024-03-27T08:41:48.781 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.787+0000 7f6013e29ec0 1 -- 172.21.15.104:0/1316873609 wait complete. 2024-03-27T08:41:48.782 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.782 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj4 /etc/passwd 2024-03-27T08:41:48.816 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.824+0000 7fe78aa31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.829+0000 7fe78aa31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.829+0000 7fe78aa31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.830+0000 7fe78aa31ec0 1 Processor -- start 2024-03-27T08:41:48.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.830+0000 7fe78aa31ec0 1 -- start start 2024-03-27T08:41:48.823 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe78aa31ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbe7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe78aa31ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xab80b0 con 0xb388f0 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe782b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbe7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe782b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbe7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47034/0 (socket says 172.21.15.104:47034) 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe782b9b640 1 -- 172.21.15.104:0/2285923618 learned_addr learned my addr 172.21.15.104:0/2285923618 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe782b9b640 1 -- 172.21.15.104:0/2285923618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xa581e0 con 0xb388f0 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe782b9b640 1 --2- 172.21.15.104:0/2285923618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbe7bf0 crc :-1 s=READY pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e941d5949d567bd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe781398640 1 -- 172.21.15.104:0/2285923618 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe7740061e0 con 0xb388f0 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe781398640 1 -- 172.21.15.104:0/2285923618 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe774006340 con 0xb388f0 2024-03-27T08:41:48.824 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.831+0000 7fe781398640 1 -- 172.21.15.104:0/2285923618 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe774004880 con 0xb388f0 2024-03-27T08:41:48.825 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2285923618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 msgr2=0xbe7bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.825 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2285923618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbe7bf0 crc :-1 s=READY pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.825 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2285923618 shutdown_connections 2024-03-27T08:41:48.825 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2285923618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbe7bf0 unknown :-1 s=CLOSED pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.826 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2285923618 >> 172.21.15.104:0/2285923618 conn(0xa6cea0 msgr2=0xb4d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.826 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2285923618 shutdown_connections 2024-03-27T08:41:48.826 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.832+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2285923618 wait complete. 2024-03-27T08:41:48.826 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.833+0000 7fe78aa31ec0 1 Processor -- start 2024-03-27T08:41:48.826 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.833+0000 7fe78aa31ec0 1 -- start start 2024-03-27T08:41:48.827 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe78aa31ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbcf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.827 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe78aa31ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xab5b30 con 0xb388f0 2024-03-27T08:41:48.827 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe782b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbcf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.827 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe782b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbcf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47044/0 (socket says 172.21.15.104:47044) 2024-03-27T08:41:48.827 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe782b9b640 1 -- 172.21.15.104:0/2507659950 learned_addr learned my addr 172.21.15.104:0/2507659950 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe782b9b640 1 -- 172.21.15.104:0/2507659950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xa4ae30 con 0xb388f0 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe782b9b640 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbcf5e0 crc :-1 s=READY pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe77b7fe640 1 -- 172.21.15.104:0/2507659950 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe774004380 con 0xb388f0 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe77b7fe640 1 -- 172.21.15.104:0/2507659950 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe774001880 con 0xb388f0 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe77b7fe640 1 -- 172.21.15.104:0/2507659950 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe774001b20 con 0xb388f0 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xa4a890 con 0xb388f0 2024-03-27T08:41:48.828 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.834+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0xbd2230 con 0xb388f0 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.835+0000 7fe77b7fe640 1 -- 172.21.15.104:0/2507659950 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe774001d20 con 0xb388f0 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.836+0000 7fe77b7fe640 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe768036fe0 0x7fe7680394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.836+0000 7fe77b7fe640 1 -- 172.21.15.104:0/2507659950 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7fe7740482a0 con 0xb388f0 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.836+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> v2:172.21.15.177:6807/929709834 conn(0xb39410 0xc7d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.836+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 200.0 200:0ee9ae15:::obj4:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0xbd30b0 con 0xb39410 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.836+0000 7fe78239a640 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe768036fe0 0x7fe7680394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.829 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.836+0000 7fe78239a640 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe768036fe0 0x7fe7680394f0 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.830 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.837+0000 7fe78339c640 1 --2- 172.21.15.104:0/2507659950 >> v2:172.21.15.177:6807/929709834 conn(0xb39410 0xc7d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:48.832 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.840+0000 7fe78339c640 1 --2- 172.21.15.104:0/2507659950 >> v2:172.21.15.177:6807/929709834 conn(0xb39410 0xc7d190 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.851+0000 7fe78339c640 1 -- 172.21.15.104:0/2507659950 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj4 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fe77000c430 con 0xb39410 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 >> v2:172.21.15.177:6807/929709834 conn(0xb39410 msgr2=0xc7d190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> v2:172.21.15.177:6807/929709834 conn(0xb39410 0xc7d190 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe768036fe0 msgr2=0x7fe7680394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe768036fe0 0x7fe7680394f0 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 msgr2=0xbcf5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbcf5e0 crc :-1 s=READY pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 shutdown_connections 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> v2:172.21.15.177:6807/929709834 conn(0xb39410 0xc7d190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe768036fe0 0x7fe7680394f0 unknown :-1 s=CLOSED pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.845 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 --2- 172.21.15.104:0/2507659950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xb388f0 0xbcf5e0 unknown :-1 s=CLOSED pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.846 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 >> 172.21.15.104:0/2507659950 conn(0xa6cea0 msgr2=0xb54dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.846 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 shutdown_connections 2024-03-27T08:41:48.846 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.852+0000 7fe78aa31ec0 1 -- 172.21.15.104:0/2507659950 wait complete. 2024-03-27T08:41:48.846 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.846 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj5 /etc/passwd 2024-03-27T08:41:48.882 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.890+0000 7f8a1782cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.887 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.895+0000 7f8a1782cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.887 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.895+0000 7f8a1782cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.888 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.895+0000 7f8a1782cec0 1 Processor -- start 2024-03-27T08:41:48.888 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.895+0000 7f8a1782cec0 1 -- start start 2024-03-27T08:41:48.889 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.896+0000 7f8a1782cec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1320be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.889 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.896+0000 7f8a1782cec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x116e0b0 con 0x1208f70 2024-03-27T08:41:48.889 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.896+0000 7f8a0affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1320be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.896+0000 7f8a0affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1320be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47066/0 (socket says 172.21.15.104:47066) 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.896+0000 7f8a0affd640 1 -- 172.21.15.104:0/2995031957 learned_addr learned my addr 172.21.15.104:0/2995031957 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.897+0000 7f8a0affd640 1 -- 172.21.15.104:0/2995031957 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x110e1e0 con 0x1208f70 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.897+0000 7f8a0affd640 1 --2- 172.21.15.104:0/2995031957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1320be0 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb879876983d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.897+0000 7f8a097fa640 1 -- 172.21.15.104:0/2995031957 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f89fc00c490 con 0x1208f70 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.897+0000 7f8a097fa640 1 -- 172.21.15.104:0/2995031957 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f89fc00c5f0 con 0x1208f70 2024-03-27T08:41:48.890 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.897+0000 7f8a097fa640 1 -- 172.21.15.104:0/2995031957 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f89fc00c890 con 0x1208f70 2024-03-27T08:41:48.891 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2995031957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 msgr2=0x1320be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.891 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2995031957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1320be0 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.891 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2995031957 shutdown_connections 2024-03-27T08:41:48.891 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2995031957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1320be0 unknown :-1 s=CLOSED pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.891 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2995031957 >> 172.21.15.104:0/2995031957 conn(0x1122ea0 msgr2=0x1324430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.892 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2995031957 shutdown_connections 2024-03-27T08:41:48.892 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2995031957 wait complete. 2024-03-27T08:41:48.892 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 Processor -- start 2024-03-27T08:41:48.892 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.898+0000 7f8a1782cec0 1 -- start start 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.899+0000 7f8a1782cec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1330be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.899+0000 7f8a1782cec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x116bb30 con 0x1208f70 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a0affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1330be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a0affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1330be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47080/0 (socket says 172.21.15.104:47080) 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a0affd640 1 -- 172.21.15.104:0/2348812581 learned_addr learned my addr 172.21.15.104:0/2348812581 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a0affd640 1 -- 172.21.15.104:0/2348812581 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1100e30 con 0x1208f70 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a0affd640 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1330be0 crc :-1 s=READY pgs=12866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f89f3fff640 1 -- 172.21.15.104:0/2348812581 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f89fc0111e0 con 0x1208f70 2024-03-27T08:41:48.894 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f89f3fff640 1 -- 172.21.15.104:0/2348812581 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f89fc011a00 con 0x1208f70 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1100890 con 0x1208f70 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f89f3fff640 1 -- 172.21.15.104:0/2348812581 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f89fc011b70 con 0x1208f70 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.900+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x130e420 con 0x1208f70 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.901+0000 7f89f3fff640 1 -- 172.21.15.104:0/2348812581 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f89fc011d20 con 0x1208f70 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.901+0000 7f89f3fff640 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f89f8037030 0x7f89f8039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.901+0000 7f89f3fff640 1 -- 172.21.15.104:0/2348812581 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f89fc048900 con 0x1208f70 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.901+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> v2:172.21.15.177:6807/929709834 conn(0x1209a90 0x130b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.901+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 200.3 200:c52c9666:::obj5:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x130ee90 con 0x1209a90 2024-03-27T08:41:48.895 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.902+0000 7f8a0a7fc640 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f89f8037030 0x7f89f8039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.896 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.902+0000 7f8a0a7fc640 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f89f8037030 0x7f89f8039540 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.896 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.902+0000 7f8a0b7fe640 1 --2- 172.21.15.104:0/2348812581 >> v2:172.21.15.177:6807/929709834 conn(0x1209a90 0x130b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:48.897 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.905+0000 7f8a0b7fe640 1 --2- 172.21.15.104:0/2348812581 >> v2:172.21.15.177:6807/929709834 conn(0x1209a90 0x130b8e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a0b7fe640 1 -- 172.21.15.104:0/2348812581 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj5 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f8a0400a430 con 0x1209a90 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 >> v2:172.21.15.177:6807/929709834 conn(0x1209a90 msgr2=0x130b8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> v2:172.21.15.177:6807/929709834 conn(0x1209a90 0x130b8e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f89f8037030 msgr2=0x7f89f8039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f89f8037030 0x7f89f8039540 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 msgr2=0x1330be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1330be0 crc :-1 s=READY pgs=12866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 shutdown_connections 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> v2:172.21.15.177:6807/929709834 conn(0x1209a90 0x130b8e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.909 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f89f8037030 0x7f89f8039540 unknown :-1 s=CLOSED pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.910 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 --2- 172.21.15.104:0/2348812581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1208f70 0x1330be0 unknown :-1 s=CLOSED pgs=12866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.910 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 >> 172.21.15.104:0/2348812581 conn(0x1122ea0 msgr2=0x1324430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.910 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 shutdown_connections 2024-03-27T08:41:48.910 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.916+0000 7f8a1782cec0 1 -- 172.21.15.104:0/2348812581 wait complete. 2024-03-27T08:41:48.911 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.911 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj6 /etc/passwd 2024-03-27T08:41:48.946 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.954+0000 7f74df2feec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.951 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.959+0000 7f74df2feec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.951 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.959+0000 7f74df2feec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:48.951 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.959+0000 7f74df2feec0 1 Processor -- start 2024-03-27T08:41:48.951 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.959+0000 7f74df2feec0 1 -- start start 2024-03-27T08:41:48.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.960+0000 7f74df2feec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x1364470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.960+0000 7f74df2feec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x11ab0b0 con 0x12d7360 2024-03-27T08:41:48.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.960+0000 7f74d779b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x1364470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.960+0000 7f74d779b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x1364470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47090/0 (socket says 172.21.15.104:47090) 2024-03-27T08:41:48.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.960+0000 7f74d779b640 1 -- 172.21.15.104:0/2381942343 learned_addr learned my addr 172.21.15.104:0/2381942343 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.953 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.961+0000 7f74d779b640 1 -- 172.21.15.104:0/2381942343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x114b1e0 con 0x12d7360 2024-03-27T08:41:48.954 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.961+0000 7f74d779b640 1 --2- 172.21.15.104:0/2381942343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x1364470 crc :-1 s=READY pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429ac1d8f1ef1cff server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.954 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.961+0000 7f74d5f98640 1 -- 172.21.15.104:0/2381942343 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74c000b500 con 0x12d7360 2024-03-27T08:41:48.954 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.961+0000 7f74d5f98640 1 -- 172.21.15.104:0/2381942343 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f74c000b660 con 0x12d7360 2024-03-27T08:41:48.954 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.961+0000 7f74d5f98640 1 -- 172.21.15.104:0/2381942343 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74c000b900 con 0x12d7360 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 -- 172.21.15.104:0/2381942343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 msgr2=0x1364470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 --2- 172.21.15.104:0/2381942343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x1364470 crc :-1 s=READY pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 -- 172.21.15.104:0/2381942343 shutdown_connections 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 --2- 172.21.15.104:0/2381942343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x1364470 unknown :-1 s=CLOSED pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 -- 172.21.15.104:0/2381942343 >> 172.21.15.104:0/2381942343 conn(0x115fea0 msgr2=0x123ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 -- 172.21.15.104:0/2381942343 shutdown_connections 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 -- 172.21.15.104:0/2381942343 wait complete. 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 Processor -- start 2024-03-27T08:41:48.955 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.962+0000 7f74df2feec0 1 -- start start 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.963+0000 7f74df2feec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x12d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.963+0000 7f74df2feec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x11a8b30 con 0x12d7360 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.963+0000 7f74d779b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x12d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.963+0000 7f74d779b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x12d51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47102/0 (socket says 172.21.15.104:47102) 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.963+0000 7f74d779b640 1 -- 172.21.15.104:0/522418548 learned_addr learned my addr 172.21.15.104:0/522418548 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74d779b640 1 -- 172.21.15.104:0/522418548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x113de30 con 0x12d7360 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74d779b640 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x12d51b0 crc :-1 s=READY pgs=12870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74d4f96640 1 -- 172.21.15.104:0/522418548 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74c00110b0 con 0x12d7360 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74d4f96640 1 -- 172.21.15.104:0/522418548 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f74c0011210 con 0x12d7360 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x113d890 con 0x12d7360 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74d4f96640 1 -- 172.21.15.104:0/522418548 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74c0011a60 con 0x12d7360 2024-03-27T08:41:48.957 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.964+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1363ed0 con 0x12d7360 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.965+0000 7f74d4f96640 1 -- 172.21.15.104:0/522418548 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f74c0011c60 con 0x12d7360 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.965+0000 7f74d4f96640 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f74bc036fe0 0x7f74bc0394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.965+0000 7f74d4f96640 1 -- 172.21.15.104:0/522418548 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f74c0048970 con 0x12d7360 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.965+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> v2:172.21.15.104:6801/1224298329 conn(0x12da190 0x136c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.965+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 200.4 200:233a42c1:::obj6:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x1364890 con 0x12da190 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.965+0000 7f74d6f9a640 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f74bc036fe0 0x7f74bc0394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.966+0000 7f74d6f9a640 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f74bc036fe0 0x7f74bc0394f0 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.959 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.966+0000 7f74d7f9c640 1 --2- 172.21.15.104:0/522418548 >> v2:172.21.15.104:6801/1224298329 conn(0x12da190 0x136c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:48.961 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.968+0000 7f74d7f9c640 1 --2- 172.21.15.104:0/522418548 >> v2:172.21.15.104:6801/1224298329 conn(0x12da190 0x136c060 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:48.972 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74d7f9c640 1 -- 172.21.15.104:0/522418548 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj6 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f74c800a430 con 0x12da190 2024-03-27T08:41:48.972 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 >> v2:172.21.15.104:6801/1224298329 conn(0x12da190 msgr2=0x136c060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.972 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> v2:172.21.15.104:6801/1224298329 conn(0x12da190 0x136c060 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.972 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f74bc036fe0 msgr2=0x7f74bc0394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.972 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f74bc036fe0 0x7f74bc0394f0 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 msgr2=0x12d51b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x12d51b0 crc :-1 s=READY pgs=12870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 shutdown_connections 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f74bc036fe0 0x7f74bc0394f0 unknown :-1 s=CLOSED pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> v2:172.21.15.104:6801/1224298329 conn(0x12da190 0x136c060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 --2- 172.21.15.104:0/522418548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x12d7360 0x12d51b0 unknown :-1 s=CLOSED pgs=12870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.979+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 >> 172.21.15.104:0/522418548 conn(0x115fea0 msgr2=0x123fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.980+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 shutdown_connections 2024-03-27T08:41:48.973 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:48.980+0000 7f74df2feec0 1 -- 172.21.15.104:0/522418548 wait complete. 2024-03-27T08:41:48.974 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:48.974 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj7 /etc/passwd 2024-03-27T08:41:49.009 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.017+0000 7f8e76662ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.014 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.022+0000 7f8e76662ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.014 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.022+0000 7f8e76662ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.014 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.022+0000 7f8e76662ec0 1 Processor -- start 2024-03-27T08:41:49.014 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.022+0000 7f8e76662ec0 1 -- start start 2024-03-27T08:41:49.016 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.023+0000 7f8e76662ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x23748a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.016 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.023+0000 7f8e76662ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x21c00b0 con 0x225aad0 2024-03-27T08:41:49.016 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.023+0000 7f8e6e79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x23748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.016 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.023+0000 7f8e6e79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x23748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47120/0 (socket says 172.21.15.104:47120) 2024-03-27T08:41:49.016 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.023+0000 7f8e6e79b640 1 -- 172.21.15.104:0/626218294 learned_addr learned my addr 172.21.15.104:0/626218294 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.017 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e6e79b640 1 -- 172.21.15.104:0/626218294 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x21601e0 con 0x225aad0 2024-03-27T08:41:49.017 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e6e79b640 1 --2- 172.21.15.104:0/626218294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x23748a0 crc :-1 s=READY pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf1d0aadbb0a16e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.017 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e6cf98640 1 -- 172.21.15.104:0/626218294 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8e60033040 con 0x225aad0 2024-03-27T08:41:49.017 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e6cf98640 1 -- 172.21.15.104:0/626218294 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8e60034040 con 0x225aad0 2024-03-27T08:41:49.017 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e6cf98640 1 -- 172.21.15.104:0/626218294 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8e6002ecb0 con 0x225aad0 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e76662ec0 1 -- 172.21.15.104:0/626218294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 msgr2=0x23748a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.024+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/626218294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x23748a0 crc :-1 s=READY pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 -- 172.21.15.104:0/626218294 shutdown_connections 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/626218294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x23748a0 unknown :-1 s=CLOSED pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 -- 172.21.15.104:0/626218294 >> 172.21.15.104:0/626218294 conn(0x2174ea0 msgr2=0x225edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 -- 172.21.15.104:0/626218294 shutdown_connections 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 -- 172.21.15.104:0/626218294 wait complete. 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 Processor -- start 2024-03-27T08:41:49.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.025+0000 7f8e76662ec0 1 -- start start 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.026+0000 7f8e76662ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x22d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.026+0000 7f8e76662ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x21bdb30 con 0x225aad0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.026+0000 7f8e6e79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x22d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.026+0000 7f8e6e79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x22d8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47136/0 (socket says 172.21.15.104:47136) 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.026+0000 7f8e6e79b640 1 -- 172.21.15.104:0/3964576 learned_addr learned my addr 172.21.15.104:0/3964576 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e6e79b640 1 -- 172.21.15.104:0/3964576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2152e30 con 0x225aad0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e6e79b640 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x22d8b70 crc :-1 s=READY pgs=12874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e66ffd640 1 -- 172.21.15.104:0/3964576 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8e60033040 con 0x225aad0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e66ffd640 1 -- 172.21.15.104:0/3964576 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8e60009750 con 0x225aad0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x2152890 con 0x225aad0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e66ffd640 1 -- 172.21.15.104:0/3964576 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8e600099f0 con 0x225aad0 2024-03-27T08:41:49.020 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.027+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x22dbdb0 con 0x225aad0 2024-03-27T08:41:49.021 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.028+0000 7f8e66ffd640 1 -- 172.21.15.104:0/3964576 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f8e60009be0 con 0x225aad0 2024-03-27T08:41:49.021 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.028+0000 7f8e66ffd640 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8e54037030 0x7f8e54039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.028+0000 7f8e66ffd640 1 -- 172.21.15.104:0/3964576 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f8e6006b6c0 con 0x225aad0 2024-03-27T08:41:49.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.028+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> v2:172.21.15.104:6804/1714017423 conn(0x22ed880 0x22ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.028+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 --> v2:172.21.15.104:6804/1714017423 -- osd_op(unknown.0.0:1 200.9 200:9a09113a:::obj7:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x22dc510 con 0x22ed880 2024-03-27T08:41:49.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.028+0000 7f8e6df9a640 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8e54037030 0x7f8e54039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.029+0000 7f8e6df9a640 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8e54037030 0x7f8e54039540 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.029+0000 7f8e6ef9c640 1 --2- 172.21.15.104:0/3964576 >> v2:172.21.15.104:6804/1714017423 conn(0x22ed880 0x22ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:49.031 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.039+0000 7f8e6ef9c640 1 --2- 172.21.15.104:0/3964576 >> v2:172.21.15.104:6804/1714017423 conn(0x22ed880 0x22ebf10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.043 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.050+0000 7f8e6ef9c640 1 -- 172.21.15.104:0/3964576 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== osd_op_reply(1 obj7 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f8e5800c430 con 0x22ed880 2024-03-27T08:41:49.043 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 >> v2:172.21.15.104:6804/1714017423 conn(0x22ed880 msgr2=0x22ebf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.043 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> v2:172.21.15.104:6804/1714017423 conn(0x22ed880 0x22ebf10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8e54037030 msgr2=0x7f8e54039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8e54037030 0x7f8e54039540 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 msgr2=0x22d8b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x22d8b70 crc :-1 s=READY pgs=12874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 shutdown_connections 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> v2:172.21.15.104:6804/1714017423 conn(0x22ed880 0x22ebf10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f8e54037030 0x7f8e54039540 unknown :-1 s=CLOSED pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 --2- 172.21.15.104:0/3964576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x225aad0 0x22d8b70 unknown :-1 s=CLOSED pgs=12874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 >> 172.21.15.104:0/3964576 conn(0x2174ea0 msgr2=0x225edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 shutdown_connections 2024-03-27T08:41:49.044 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.051+0000 7f8e76662ec0 1 -- 172.21.15.104:0/3964576 wait complete. 2024-03-27T08:41:49.046 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:49.046 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj8 /etc/passwd 2024-03-27T08:41:49.080 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.088+0000 7f6ce2efaec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.085 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.093+0000 7f6ce2efaec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.085 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.093+0000 7f6ce2efaec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.094+0000 7f6ce2efaec0 1 Processor -- start 2024-03-27T08:41:49.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.094+0000 7f6ce2efaec0 1 -- start start 2024-03-27T08:41:49.087 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6ce2efaec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x23b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6ce2efaec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x23210b0 con 0x23b7c90 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cdb39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x23b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cdb39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x23b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47152/0 (socket says 172.21.15.104:47152) 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cdb39b640 1 -- 172.21.15.104:0/2183779356 learned_addr learned my addr 172.21.15.104:0/2183779356 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cdb39b640 1 -- 172.21.15.104:0/2183779356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x22c11e0 con 0x23b7c90 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cdb39b640 1 --2- 172.21.15.104:0/2183779356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x23b58c0 crc :-1 s=READY pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97987facdb6cd60d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cd9b98640 1 -- 172.21.15.104:0/2183779356 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc40060b0 con 0x23b7c90 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cd9b98640 1 -- 172.21.15.104:0/2183779356 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6cc4006210 con 0x23b7c90 2024-03-27T08:41:49.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.095+0000 7f6cd9b98640 1 -- 172.21.15.104:0/2183779356 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc400d960 con 0x23b7c90 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/2183779356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 msgr2=0x23b58c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/2183779356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x23b58c0 crc :-1 s=READY pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/2183779356 shutdown_connections 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/2183779356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x23b58c0 unknown :-1 s=CLOSED pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/2183779356 >> 172.21.15.104:0/2183779356 conn(0x22d5ea0 msgr2=0x2452040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/2183779356 shutdown_connections 2024-03-27T08:41:49.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.096+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/2183779356 wait complete. 2024-03-27T08:41:49.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.097+0000 7f6ce2efaec0 1 Processor -- start 2024-03-27T08:41:49.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.097+0000 7f6ce2efaec0 1 -- start start 2024-03-27T08:41:49.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6ce2efaec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x244d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6ce2efaec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x231eb30 con 0x23b7c90 2024-03-27T08:41:49.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cdb39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x244d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cdb39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x244d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47160/0 (socket says 172.21.15.104:47160) 2024-03-27T08:41:49.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cdb39b640 1 -- 172.21.15.104:0/89148587 learned_addr learned my addr 172.21.15.104:0/89148587 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cdb39b640 1 -- 172.21.15.104:0/89148587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x22b3e30 con 0x23b7c90 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cdb39b640 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x244d600 crc :-1 s=READY pgs=12878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cd8b96640 1 -- 172.21.15.104:0/89148587 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc4017040 con 0x23b7c90 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cd8b96640 1 -- 172.21.15.104:0/89148587 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6cc4006380 con 0x23b7c90 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x22b3890 con 0x23b7c90 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6cd8b96640 1 -- 172.21.15.104:0/89148587 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc4010440 con 0x23b7c90 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.098+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x24dc910 con 0x23b7c90 2024-03-27T08:41:49.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.099+0000 7f6cd8b96640 1 -- 172.21.15.104:0/89148587 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f6cc40105f0 con 0x23b7c90 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.099+0000 7f6cd8b96640 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6cc0037030 0x7f6cc0039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.099+0000 7f6cd8b96640 1 -- 172.21.15.104:0/89148587 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f6cc404a700 con 0x23b7c90 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.100+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> v2:172.21.15.104:6804/1714017423 conn(0x23b9880 0x24db360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.100+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 --> v2:172.21.15.104:6804/1714017423 -- osd_op(unknown.0.0:1 200.5 200:a557efb1:::obj8:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x2437740 con 0x23b9880 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.100+0000 7f6cdab9a640 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6cc0037030 0x7f6cc0039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.100+0000 7f6cdab9a640 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6cc0037030 0x7f6cc0039540 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.101+0000 7f6cdbb9c640 1 --2- 172.21.15.104:0/89148587 >> v2:172.21.15.104:6804/1714017423 conn(0x23b9880 0x24db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:49.096 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.104+0000 7f6cdbb9c640 1 --2- 172.21.15.104:0/89148587 >> v2:172.21.15.104:6804/1714017423 conn(0x23b9880 0x24db360 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6cdbb9c640 1 -- 172.21.15.104:0/89148587 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== osd_op_reply(1 obj8 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f6ccc00c430 con 0x23b9880 2024-03-27T08:41:49.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 >> v2:172.21.15.104:6804/1714017423 conn(0x23b9880 msgr2=0x24db360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> v2:172.21.15.104:6804/1714017423 conn(0x23b9880 0x24db360 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6cc0037030 msgr2=0x7f6cc0039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6cc0037030 0x7f6cc0039540 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 msgr2=0x244d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x244d600 crc :-1 s=READY pgs=12878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 shutdown_connections 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> v2:172.21.15.104:6804/1714017423 conn(0x23b9880 0x24db360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6cc0037030 0x7f6cc0039540 unknown :-1 s=CLOSED pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 --2- 172.21.15.104:0/89148587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x23b7c90 0x244d600 unknown :-1 s=CLOSED pgs=12878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 >> 172.21.15.104:0/89148587 conn(0x22d5ea0 msgr2=0x2452040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 shutdown_connections 2024-03-27T08:41:49.109 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.115+0000 7f6ce2efaec0 1 -- 172.21.15.104:0/89148587 wait complete. 2024-03-27T08:41:49.110 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:49.110 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj9 /etc/passwd 2024-03-27T08:41:49.145 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.152+0000 7fbca7631ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.157+0000 7fbca7631ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.157+0000 7fbca7631ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.150 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.158+0000 7fbca7631ec0 1 Processor -- start 2024-03-27T08:41:49.150 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.158+0000 7fbca7631ec0 1 -- start start 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbca7631ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x105e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbca7631ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xfca0b0 con 0x1060c90 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9f79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x105e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9f79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x105e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47170/0 (socket says 172.21.15.104:47170) 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9f79b640 1 -- 172.21.15.104:0/4254022811 learned_addr learned my addr 172.21.15.104:0/4254022811 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9f79b640 1 -- 172.21.15.104:0/4254022811 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xf6a1e0 con 0x1060c90 2024-03-27T08:41:49.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9f79b640 1 --2- 172.21.15.104:0/4254022811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x105e8c0 crc :-1 s=READY pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4ff3cc84de7b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9df98640 1 -- 172.21.15.104:0/4254022811 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbc8800b500 con 0x1060c90 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9df98640 1 -- 172.21.15.104:0/4254022811 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbc8800b660 con 0x1060c90 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.159+0000 7fbc9df98640 1 -- 172.21.15.104:0/4254022811 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbc8800b900 con 0x1060c90 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 -- 172.21.15.104:0/4254022811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 msgr2=0x105e8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/4254022811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x105e8c0 crc :-1 s=READY pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 -- 172.21.15.104:0/4254022811 shutdown_connections 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/4254022811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x105e8c0 unknown :-1 s=CLOSED pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 -- 172.21.15.104:0/4254022811 >> 172.21.15.104:0/4254022811 conn(0xf7eea0 msgr2=0x10fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 -- 172.21.15.104:0/4254022811 shutdown_connections 2024-03-27T08:41:49.155 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.160+0000 7fbca7631ec0 1 -- 172.21.15.104:0/4254022811 wait complete. 2024-03-27T08:41:49.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.161+0000 7fbca7631ec0 1 Processor -- start 2024-03-27T08:41:49.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.161+0000 7fbca7631ec0 1 -- start start 2024-03-27T08:41:49.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.162+0000 7fbca7631ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x1183e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.162+0000 7fbca7631ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xfc7b30 con 0x1060c90 2024-03-27T08:41:49.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.162+0000 7fbc9f79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x1183e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.162+0000 7fbc9f79b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x1183e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47174/0 (socket says 172.21.15.104:47174) 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.162+0000 7fbc9f79b640 1 -- 172.21.15.104:0/3809102406 learned_addr learned my addr 172.21.15.104:0/3809102406 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbc9f79b640 1 -- 172.21.15.104:0/3809102406 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xf5ce30 con 0x1060c90 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbc9f79b640 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x1183e40 crc :-1 s=READY pgs=12882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbc9cf96640 1 -- 172.21.15.104:0/3809102406 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbc880110b0 con 0x1060c90 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbc9cf96640 1 -- 172.21.15.104:0/3809102406 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbc88011210 con 0x1060c90 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xf5c890 con 0x1060c90 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbc9cf96640 1 -- 172.21.15.104:0/3809102406 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbc88011ab0 con 0x1060c90 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.163+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x10e0810 con 0x1060c90 2024-03-27T08:41:49.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.164+0000 7fbc9cf96640 1 -- 172.21.15.104:0/3809102406 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fbc88011ca0 con 0x1060c90 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.164+0000 7fbc9cf96640 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fbc78039240 0x7fbc7803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.164+0000 7fbc9cf96640 1 -- 172.21.15.104:0/3809102406 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7fbc880488a0 con 0x1060c90 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.164+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> v2:172.21.15.177:6807/929709834 conn(0x1062880 0x10f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.164+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 200.7 200:fe351e27:::obj9:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x1187060 con 0x1062880 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.164+0000 7fbc9ef9a640 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fbc78039240 0x7fbc7803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.165+0000 7fbc9ef9a640 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fbc78039240 0x7fbc7803b750 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.165+0000 7fbc9ff9c640 1 --2- 172.21.15.104:0/3809102406 >> v2:172.21.15.177:6807/929709834 conn(0x1062880 0x10f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:49.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.169+0000 7fbc9ff9c640 1 --2- 172.21.15.104:0/3809102406 >> v2:172.21.15.177:6807/929709834 conn(0x1062880 0x10f5fc0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbc9ff9c640 1 -- 172.21.15.104:0/3809102406 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj9 [writefull 0~11812] v686'2 uv2 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbc9000a430 con 0x1062880 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 >> v2:172.21.15.177:6807/929709834 conn(0x1062880 msgr2=0x10f5fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> v2:172.21.15.177:6807/929709834 conn(0x1062880 0x10f5fc0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fbc78039240 msgr2=0x7fbc7803b750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fbc78039240 0x7fbc7803b750 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 msgr2=0x1183e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x1183e40 crc :-1 s=READY pgs=12882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 shutdown_connections 2024-03-27T08:41:49.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> v2:172.21.15.177:6807/929709834 conn(0x1062880 0x10f5fc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fbc78039240 0x7fbc7803b750 unknown :-1 s=CLOSED pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 --2- 172.21.15.104:0/3809102406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1060c90 0x1183e40 unknown :-1 s=CLOSED pgs=12882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 >> 172.21.15.104:0/3809102406 conn(0xf7eea0 msgr2=0x10fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 shutdown_connections 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.180+0000 7fbca7631ec0 1 -- 172.21.15.104:0/3809102406 wait complete. 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:41:49.174 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put obj10 /etc/passwd 2024-03-27T08:41:49.209 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.217+0000 7f61f78dcec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.214 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.222+0000 7f61f78dcec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.214 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.222+0000 7f61f78dcec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:49.214 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.222+0000 7f61f78dcec0 1 Processor -- start 2024-03-27T08:41:49.215 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.222+0000 7f61f78dcec0 1 -- start start 2024-03-27T08:41:49.216 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.223+0000 7f61f78dcec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x248b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.216 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.223+0000 7f61f78dcec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x23f60b0 con 0x24768f0 2024-03-27T08:41:49.216 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.223+0000 7f61eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x248b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.223+0000 7f61eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x248b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47196/0 (socket says 172.21.15.104:47196) 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.223+0000 7f61eaffd640 1 -- 172.21.15.104:0/689978453 learned_addr learned my addr 172.21.15.104:0/689978453 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.224+0000 7f61eaffd640 1 -- 172.21.15.104:0/689978453 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x23961e0 con 0x24768f0 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.224+0000 7f61eaffd640 1 --2- 172.21.15.104:0/689978453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x248b1a0 crc :-1 s=READY pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979232301ff378e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.224+0000 7f61e97fa640 1 -- 172.21.15.104:0/689978453 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f61e0029c00 con 0x24768f0 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.224+0000 7f61e97fa640 1 -- 172.21.15.104:0/689978453 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f61e000a040 con 0x24768f0 2024-03-27T08:41:49.217 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.224+0000 7f61e97fa640 1 -- 172.21.15.104:0/689978453 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f61e000a2e0 con 0x24768f0 2024-03-27T08:41:49.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 -- 172.21.15.104:0/689978453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 msgr2=0x248b1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/689978453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x248b1a0 crc :-1 s=READY pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 -- 172.21.15.104:0/689978453 shutdown_connections 2024-03-27T08:41:49.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/689978453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x248b1a0 unknown :-1 s=CLOSED pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 -- 172.21.15.104:0/689978453 >> 172.21.15.104:0/689978453 conn(0x23aaea0 msgr2=0x2524080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 -- 172.21.15.104:0/689978453 shutdown_connections 2024-03-27T08:41:49.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 -- 172.21.15.104:0/689978453 wait complete. 2024-03-27T08:41:49.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 Processor -- start 2024-03-27T08:41:49.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.225+0000 7f61f78dcec0 1 -- start start 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.226+0000 7f61f78dcec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x2528200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.226+0000 7f61f78dcec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x23f3b30 con 0x24768f0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x2528200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x2528200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47212/0 (socket says 172.21.15.104:47212) 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61eaffd640 1 -- 172.21.15.104:0/3186494455 learned_addr learned my addr 172.21.15.104:0/3186494455 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61eaffd640 1 -- 172.21.15.104:0/3186494455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2388e30 con 0x24768f0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61eaffd640 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x2528200 crc :-1 s=READY pgs=12886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61d3fff640 1 -- 172.21.15.104:0/3186494455 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f61e003b040 con 0x24768f0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61d3fff640 1 -- 172.21.15.104:0/3186494455 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f61e00369a0 con 0x24768f0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61d3fff640 1 -- 172.21.15.104:0/3186494455 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f61e0036c40 con 0x24768f0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x2388890 con 0x24768f0 2024-03-27T08:41:49.221 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.227+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2527cb0 con 0x24768f0 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.228+0000 7f61d3fff640 1 -- 172.21.15.104:0/3186494455 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f61e003e040 con 0x24768f0 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.228+0000 7f61d3fff640 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f61d8036fe0 0x7f61d80394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.228+0000 7f61d3fff640 1 -- 172.21.15.104:0/3186494455 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(686..686 src has 397..686) ==== 3791+0+0 (crc 0 0 0) 0x7f61e002e040 con 0x24768f0 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.228+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> v2:172.21.15.104:6804/1714017423 conn(0x2477240 0x25bb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.228+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 --> v2:172.21.15.104:6804/1714017423 -- osd_op(unknown.0.0:1 200.b 200:d1337354:::obj10:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) -- 0x250e110 con 0x2477240 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.228+0000 7f61ea7fc640 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f61d8036fe0 0x7f61d80394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:49.222 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.229+0000 7f61ea7fc640 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f61d8036fe0 0x7f61d80394f0 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.223 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.229+0000 7f61eb7fe640 1 --2- 172.21.15.104:0/3186494455 >> v2:172.21.15.104:6804/1714017423 conn(0x2477240 0x25bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:49.225 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.233+0000 7f61eb7fe640 1 --2- 172.21.15.104:0/3186494455 >> v2:172.21.15.104:6804/1714017423 conn(0x2477240 0x25bb450 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:49.237 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61eb7fe640 1 -- 172.21.15.104:0/3186494455 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== osd_op_reply(1 obj10 [writefull 0~11812] v686'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f61e400a430 con 0x2477240 2024-03-27T08:41:49.237 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 >> v2:172.21.15.104:6804/1714017423 conn(0x2477240 msgr2=0x25bb450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.237 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> v2:172.21.15.104:6804/1714017423 conn(0x2477240 0x25bb450 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.237 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f61d8036fe0 msgr2=0x7f61d80394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f61d8036fe0 0x7f61d80394f0 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 msgr2=0x2528200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x2528200 crc :-1 s=READY pgs=12886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 shutdown_connections 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> v2:172.21.15.104:6804/1714017423 conn(0x2477240 0x25bb450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f61d8036fe0 0x7f61d80394f0 unknown :-1 s=CLOSED pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 --2- 172.21.15.104:0/3186494455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x24768f0 0x2528200 unknown :-1 s=CLOSED pgs=12886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 >> 172.21.15.104:0/3186494455 conn(0x23aaea0 msgr2=0x2523d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 shutdown_connections 2024-03-27T08:41:49.238 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:41:49.244+0000 7f61f78dcec0 1 -- 172.21.15.104:0/3186494455 wait complete. 2024-03-27T08:41:49.239 INFO:tasks.workunit.client.0.smithi104.stderr:+ sleep 30 2024-03-27T08:42:19.242 INFO:tasks.workunit.client.0.smithi104.stderr:+ pid=99615 2024-03-27T08:42:19.242 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_objects 100 2024-03-27T08:42:19.242 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put onemore /etc/passwd 2024-03-27T08:42:19.286 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.293+0000 7f3533c30ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:19.291 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.299+0000 7f3533c30ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:19.291 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.299+0000 7f3533c30ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:19.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.300+0000 7f3533c30ec0 1 Processor -- start 2024-03-27T08:42:19.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.300+0000 7f3533c30ec0 1 -- start start 2024-03-27T08:42:19.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3533c30ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb01440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3533c30ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xa6a0b0 con 0xaea8f0 2024-03-27T08:42:19.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3526ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb01440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:19.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3526ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb01440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50830/0 (socket says 172.21.15.104:50830) 2024-03-27T08:42:19.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3526ffd640 1 -- 172.21.15.104:0/307153288 learned_addr learned my addr 172.21.15.104:0/307153288 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:19.295 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3526ffd640 1 -- 172.21.15.104:0/307153288 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xa0a1e0 con 0xaea8f0 2024-03-27T08:42:19.295 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f3526ffd640 1 --2- 172.21.15.104:0/307153288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb01440 crc :-1 s=READY pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9d316cc9bd7ec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.295 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f35257fa640 1 -- 172.21.15.104:0/307153288 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f35200061e0 con 0xaea8f0 2024-03-27T08:42:19.295 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f35257fa640 1 -- 172.21.15.104:0/307153288 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3520001940 con 0xaea8f0 2024-03-27T08:42:19.295 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.301+0000 7f35257fa640 1 -- 172.21.15.104:0/307153288 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3520006340 con 0xaea8f0 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.302+0000 7f3533c30ec0 1 -- 172.21.15.104:0/307153288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 msgr2=0xb01440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.302+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/307153288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb01440 crc :-1 s=READY pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.302+0000 7f3533c30ec0 1 -- 172.21.15.104:0/307153288 shutdown_connections 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.303+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/307153288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb01440 unknown :-1 s=CLOSED pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.303+0000 7f3533c30ec0 1 -- 172.21.15.104:0/307153288 >> 172.21.15.104:0/307153288 conn(0xa1eea0 msgr2=0xaff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.303+0000 7f3533c30ec0 1 -- 172.21.15.104:0/307153288 shutdown_connections 2024-03-27T08:42:19.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.303+0000 7f3533c30ec0 1 -- 172.21.15.104:0/307153288 wait complete. 2024-03-27T08:42:19.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.304+0000 7f3533c30ec0 1 Processor -- start 2024-03-27T08:42:19.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.304+0000 7f3533c30ec0 1 -- start start 2024-03-27T08:42:19.299 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.305+0000 7f3533c30ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb849a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.299 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.305+0000 7f3533c30ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xa67b30 con 0xaea8f0 2024-03-27T08:42:19.299 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.305+0000 7f3526ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:19.299 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.305+0000 7f3526ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb849a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50838/0 (socket says 172.21.15.104:50838) 2024-03-27T08:42:19.299 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.305+0000 7f3526ffd640 1 -- 172.21.15.104:0/2833893835 learned_addr learned my addr 172.21.15.104:0/2833893835 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.305+0000 7f3526ffd640 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x9fce30 con 0xaea8f0 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.306+0000 7f3526ffd640 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb849a0 crc :-1 s=READY pgs=12890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.306+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3520004920 con 0xaea8f0 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.306+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x9fc890 con 0xaea8f0 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.306+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3520004a80 con 0xaea8f0 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.306+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3520004d20 con 0xaea8f0 2024-03-27T08:42:19.300 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.306+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0xb87b00 con 0xaea8f0 2024-03-27T08:42:19.301 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.307+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f3520014410 con 0xaea8f0 2024-03-27T08:42:19.301 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.307+0000 7f351ffff640 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3508036fe0 0x7f35080394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.307+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(688..688 src has 397..688) ==== 3791+0+0 (crc 0 0 0) 0x7f35200484e0 con 0xaea8f0 2024-03-27T08:42:19.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.307+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=689}) -- 0x7f350803d1f0 con 0xaea8f0 2024-03-27T08:42:19.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.307+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> v2:172.21.15.104:6804/1714017423 conn(0xaeb410 0xc31220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.307+0000 7f35267fc640 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3508036fe0 0x7f35080394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:19.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.308+0000 7f35267fc640 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3508036fe0 0x7f35080394f0 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.308+0000 7f35277fe640 1 --2- 172.21.15.104:0/2833893835 >> v2:172.21.15.104:6804/1714017423 conn(0xaeb410 0xc31220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:42:19.304 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.311+0000 7f35277fe640 1 --2- 172.21.15.104:0/2833893835 >> v2:172.21.15.104:6804/1714017423 conn(0xaeb410 0xc31220 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.347+0000 7f1662a64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:19.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.357+0000 7f1662a64640 1 Processor -- start 2024-03-27T08:42:19.350 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.357+0000 7f1662a64640 1 -- start start 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.358+0000 7f1662a64640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f165c137b10 0x7f165c139f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.358+0000 7f1662a64640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165c06eb50 con 0x7f165c09f6e0 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.358+0000 7f1662a64640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f165c09f2f0 con 0x7f165c137b10 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.358+0000 7f165bfff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f165c137b10 0x7f165c139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.358+0000 7f1660800640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f165c09f6e0 0x7f165c09faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50262/0 (socket says 172.21.15.104:50262) 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.358+0000 7f1660800640 1 -- 172.21.15.104:0/842609570 learned_addr learned my addr 172.21.15.104:0/842609570 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165bfff640 1 -- 172.21.15.104:0/842609570 >> v1:172.21.15.104:6789/0 conn(0x7f165c09f6e0 legacy=0x7f165c09faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165b7fe640 1 -- 172.21.15.104:0/842609570 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1211565563 0 0) 0x7f165c06eb50 con 0x7f165c09f6e0 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165bfff640 1 -- 172.21.15.104:0/842609570 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c0a0200 con 0x7f165c137b10 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165bfff640 1 --2- 172.21.15.104:0/842609570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c137b10 0x7f165c139f30 crc :-1 s=READY pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a466c7dc1e04370 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165b7fe640 1 -- 172.21.15.104:0/842609570 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1650009d90 con 0x7f165c137b10 2024-03-27T08:42:19.352 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165b7fe640 1 -- 172.21.15.104:0/842609570 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f165000e230 con 0x7f165c137b10 2024-03-27T08:42:19.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.359+0000 7f165b7fe640 1 -- 172.21.15.104:0/842609570 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f165000e4d0 con 0x7f165c137b10 2024-03-27T08:42:19.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 -- 172.21.15.104:0/842609570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c137b10 msgr2=0x7f165c139f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 --2- 172.21.15.104:0/842609570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c137b10 0x7f165c139f30 crc :-1 s=READY pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 -- 172.21.15.104:0/842609570 shutdown_connections 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 --2- 172.21.15.104:0/842609570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c137b10 0x7f165c139f30 unknown :-1 s=CLOSED pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 -- 172.21.15.104:0/842609570 >> 172.21.15.104:0/842609570 conn(0x7f165c0a1860 msgr2=0x7f165c0a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 -- 172.21.15.104:0/842609570 shutdown_connections 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 1 -- 172.21.15.104:0/842609570 wait complete. 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.360+0000 7f1662a64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:19.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.361+0000 7f1662a64640 1 Processor -- start 2024-03-27T08:42:19.355 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.361+0000 7f1662a64640 1 -- start start 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.362+0000 7f1662a64640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 0x7f165c11f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.362+0000 7f1662a64640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f165c120760 con 0x7f165c09f6e0 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1660800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 0x7f165c11f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1660800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 0x7f165c11f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50862/0 (socket says 172.21.15.104:50862) 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1660800640 1 -- 172.21.15.104:0/4013352918 learned_addr learned my addr 172.21.15.104:0/4013352918 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1660800640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c11f120 con 0x7f165c09f6e0 2024-03-27T08:42:19.356 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1660800640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 0x7f165c11f670 crc :-1 s=READY pgs=12893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.357 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1648004b80 con 0x7f165c09f6e0 2024-03-27T08:42:19.357 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1648004ce0 con 0x7f165c09f6e0 2024-03-27T08:42:19.357 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1648007120 con 0x7f165c09f6e0 2024-03-27T08:42:19.357 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c11ed80 con 0x7f165c09f6e0 2024-03-27T08:42:19.357 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.363+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c122760 con 0x7f165c09f6e0 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.364+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f16480072d0 con 0x7f165c09f6e0 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.364+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f165c11ef70 con 0x7f165c09f6e0 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.364+0000 7f16597fa640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f16400387a0 0x7f164003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.364+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(688..688 src has 397..688) ==== 3791+0+0 (crc 0 0 0) 0x7f1648048650 con 0x7f165c09f6e0 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.365+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=689}) -- 0x7f164003e9b0 con 0x7f165c09f6e0 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.365+0000 7f165bfff640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f16400387a0 0x7f164003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:19.358 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.365+0000 7f165bfff640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f16400387a0 0x7f164003acb0 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:19.362 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.370+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f164800f230 con 0x7f165c09f6e0 2024-03-27T08:42:19.543 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:19.550+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "100"} v 0) -- 0x7f165c0693c0 con 0x7f165c09f6e0 2024-03-27T08:42:20.019 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:20.026+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v689) ==== 225+0+0 (crc 0 0 0) 0x7f164800a5d0 con 0x7f165c09f6e0 2024-03-27T08:42:20.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:20.029+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 6 ==== osd_map(689..689 src has 397..689) ==== 628+0+0 (crc 0 0 0) 0x7f352000fbf0 con 0xaea8f0 2024-03-27T08:42:20.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:20.029+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(689..689 src has 397..689) ==== 628+0+0 (crc 0 0 0) 0x7f1648006550 con 0x7f165c09f6e0 2024-03-27T08:42:20.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:20.029+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=690}) -- 0x7f164003fa80 con 0x7f165c09f6e0 2024-03-27T08:42:20.022 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:20.029+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=690}) -- 0x7f350803d880 con 0xaea8f0 2024-03-27T08:42:20.186 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:20.193+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "100"} v 0) -- 0x7f165c120fd0 con 0x7f165c09f6e0 2024-03-27T08:42:21.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.036+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v690) ==== 225+0+0 (crc 0 0 0) 0x7f164800a7b0 con 0x7f165c09f6e0 2024-03-27T08:42:21.029 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_objects = 100 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:42:21.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.036+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 <== mon.0 v2:172.21.15.104:3300/0 10 ==== osd_map(690..690 src has 397..690) ==== 628+0+0 (crc 0 0 0) 0x7f164800ac10 con 0x7f165c09f6e0 2024-03-27T08:42:21.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.036+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 7 ==== osd_map(690..690 src has 397..690) ==== 628+0+0 (crc 0 0 0) 0x7f3520014c30 con 0xaea8f0 2024-03-27T08:42:21.030 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.036+0000 7f16597fa640 1 -- 172.21.15.104:0/4013352918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=691}) -- 0x7f164003fcc0 con 0x7f165c09f6e0 2024-03-27T08:42:21.030 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.036+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=691}) -- 0x7f350803daa0 con 0xaea8f0 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f16400387a0 msgr2=0x7f164003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f16400387a0 0x7f164003acb0 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 msgr2=0x7f165c11f670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 0x7f165c11f670 crc :-1 s=READY pgs=12893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 shutdown_connections 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f16400387a0 0x7f164003acb0 unknown :-1 s=CLOSED pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 --2- 172.21.15.104:0/4013352918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f165c09f6e0 0x7f165c11f670 unknown :-1 s=CLOSED pgs=12893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 >> 172.21.15.104:0/4013352918 conn(0x7f165c0a1860 msgr2=0x7f165c134610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:21.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 shutdown_connections 2024-03-27T08:42:21.034 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:21.039+0000 7f1662a64640 1 -- 172.21.15.104:0/4013352918 wait complete. 2024-03-27T08:42:21.046 INFO:tasks.workunit.client.0.smithi104.stderr:+ wait 99615 2024-03-27T08:42:22.000 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.007+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(691..691 src has 397..691) ==== 628+0+0 (crc 0 0 0) 0x7f352000a0d0 con 0xaea8f0 2024-03-27T08:42:22.000 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.007+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=692}) -- 0x7f350803daa0 con 0xaea8f0 2024-03-27T08:42:22.000 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.007+0000 7f351ffff640 1 -- 172.21.15.104:0/2833893835 --> v2:172.21.15.104:6804/1714017423 -- osd_op(unknown.0.0:1 200.5 200:b9ff7440:::onemore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e691) -- 0x7f350803dd20 con 0xaeb410 2024-03-27T08:42:22.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f35277fe640 1 -- 172.21.15.104:0/2833893835 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== osd_op_reply(1 onemore [writefull 0~11812] v691'2 uv2 ondisk = 0) ==== 151+0+0 (crc 0 0 0) 0x7f351800c430 con 0xaeb410 2024-03-27T08:42:22.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 >> v2:172.21.15.104:6804/1714017423 conn(0xaeb410 msgr2=0xc31220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> v2:172.21.15.104:6804/1714017423 conn(0xaeb410 0xc31220 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3508036fe0 msgr2=0x7f35080394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.277 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3508036fe0 0x7f35080394f0 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 msgr2=0xb849a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.283+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb849a0 crc :-1 s=READY pgs=12890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 shutdown_connections 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> v2:172.21.15.104:6804/1714017423 conn(0xaeb410 0xc31220 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3508036fe0 0x7f35080394f0 unknown :-1 s=CLOSED pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 --2- 172.21.15.104:0/2833893835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xaea8f0 0xb849a0 unknown :-1 s=CLOSED pgs=12890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 >> 172.21.15.104:0/2833893835 conn(0xa1eea0 msgr2=0xb06dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 shutdown_connections 2024-03-27T08:42:22.278 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.284+0000 7f3533c30ec0 1 -- 172.21.15.104:0/2833893835 wait complete. 2024-03-27T08:42:22.279 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -ne 0 ']' 2024-03-27T08:42:22.279 INFO:tasks.workunit.client.0.smithi104.stderr:+ true 2024-03-27T08:42:22.279 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put twomore /etc/passwd 2024-03-27T08:42:22.314 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.321+0000 7f2a28cd0ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:22.319 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.326+0000 7f2a28cd0ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:22.319 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.326+0000 7f2a28cd0ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:22.319 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.327+0000 7f2a28cd0ec0 1 Processor -- start 2024-03-27T08:42:22.319 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.327+0000 7f2a28cd0ec0 1 -- start start 2024-03-27T08:42:22.321 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a28cd0ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x11272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.321 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a28cd0ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xff60b0 con 0x1090f70 2024-03-27T08:42:22.321 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a2119b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x11272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:22.321 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a2119b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x11272a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50888/0 (socket says 172.21.15.104:50888) 2024-03-27T08:42:22.321 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a2119b640 1 -- 172.21.15.104:0/2288047215 learned_addr learned my addr 172.21.15.104:0/2288047215 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:22.322 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a2119b640 1 -- 172.21.15.104:0/2288047215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xf961e0 con 0x1090f70 2024-03-27T08:42:22.322 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a2119b640 1 --2- 172.21.15.104:0/2288047215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x11272a0 crc :-1 s=READY pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df102f8d43069cb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.322 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a1affd640 1 -- 172.21.15.104:0/2288047215 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a140061e0 con 0x1090f70 2024-03-27T08:42:22.322 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a1affd640 1 -- 172.21.15.104:0/2288047215 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2a140019c0 con 0x1090f70 2024-03-27T08:42:22.322 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.328+0000 7f2a1affd640 1 -- 172.21.15.104:0/2288047215 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a14006340 con 0x1090f70 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/2288047215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 msgr2=0x11272a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/2288047215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x11272a0 crc :-1 s=READY pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/2288047215 shutdown_connections 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/2288047215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x11272a0 unknown :-1 s=CLOSED pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/2288047215 >> 172.21.15.104:0/2288047215 conn(0xfaaea0 msgr2=0x1087ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/2288047215 shutdown_connections 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.329+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/2288047215 wait complete. 2024-03-27T08:42:22.323 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.330+0000 7f2a28cd0ec0 1 Processor -- start 2024-03-27T08:42:22.324 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.330+0000 7f2a28cd0ec0 1 -- start start 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a28cd0ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x110e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a28cd0ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xff3b30 con 0x1090f70 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a2119b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x110e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a2119b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x110e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50898/0 (socket says 172.21.15.104:50898) 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a2119b640 1 -- 172.21.15.104:0/3385106010 learned_addr learned my addr 172.21.15.104:0/3385106010 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a2119b640 1 -- 172.21.15.104:0/3385106010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xf88e30 con 0x1090f70 2024-03-27T08:42:22.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a2119b640 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x110e540 crc :-1 s=READY pgs=12897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.331+0000 7f2a19ffb640 1 -- 172.21.15.104:0/3385106010 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a14004b90 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.332+0000 7f2a19ffb640 1 -- 172.21.15.104:0/3385106010 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2a14004cf0 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.332+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xf88890 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.332+0000 7f2a19ffb640 1 -- 172.21.15.104:0/3385106010 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a14012330 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.332+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x11b7c10 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.332+0000 7f2a19ffb640 1 -- 172.21.15.104:0/3385106010 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2a14012520 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.333+0000 7f2a19ffb640 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2a08037030 0x7f2a08039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.333+0000 7f2a19ffb640 1 -- 172.21.15.104:0/3385106010 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(691..691 src has 397..691) ==== 3791+0+0 (crc 0 0 0) 0x7f2a14048500 con 0x1090f70 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.333+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> v2:172.21.15.104:6801/1224298329 conn(0x1091a90 0x110bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.333+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 200.6 200:7019d845:::twomore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e691) -- 0x110c0f0 con 0x1091a90 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.333+0000 7f2a2099a640 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2a08037030 0x7f2a08039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:22.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.333+0000 7f2a2099a640 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2a08037030 0x7f2a08039540 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.328 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.334+0000 7f2a2199c640 1 --2- 172.21.15.104:0/3385106010 >> v2:172.21.15.104:6801/1224298329 conn(0x1091a90 0x110bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:42:22.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.337+0000 7f2a2199c640 1 --2- 172.21.15.104:0/3385106010 >> v2:172.21.15.104:6801/1224298329 conn(0x1091a90 0x110bbb0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.491 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a2199c640 1 -- 172.21.15.104:0/3385106010 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 twomore [writefull 0~11812] v691'1 uv1 ondisk = 0) ==== 151+0+0 (crc 0 0 0) 0x7f2a0c00c430 con 0x1091a90 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 >> v2:172.21.15.104:6801/1224298329 conn(0x1091a90 msgr2=0x110bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> v2:172.21.15.104:6801/1224298329 conn(0x1091a90 0x110bbb0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2a08037030 msgr2=0x7f2a08039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2a08037030 0x7f2a08039540 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 msgr2=0x110e540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x110e540 crc :-1 s=READY pgs=12897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 shutdown_connections 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2a08037030 0x7f2a08039540 unknown :-1 s=CLOSED pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> v2:172.21.15.104:6801/1224298329 conn(0x1091a90 0x110bbb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 --2- 172.21.15.104:0/3385106010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1090f70 0x110e540 unknown :-1 s=CLOSED pgs=12897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.492 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 >> 172.21.15.104:0/3385106010 conn(0xfaaea0 msgr2=0x1087ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:22.493 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 shutdown_connections 2024-03-27T08:42:22.493 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.498+0000 7f2a28cd0ec0 1 -- 172.21.15.104:0/3385106010 wait complete. 2024-03-27T08:42:22.493 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_bytes 100 2024-03-27T08:42:22.585 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.584+0000 7f4792a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:22.585 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.593+0000 7f4792a2d640 1 Processor -- start 2024-03-27T08:42:22.586 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.593+0000 7f4792a2d640 1 -- start start 2024-03-27T08:42:22.587 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f4792a2d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f478c09bea0 0x7f478c1329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f4792a2d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478c132580 con 0x7f478c132f10 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f4792a2d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f478c070bf0 con 0x7f478c09bea0 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f478bfff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f478c09bea0 0x7f478c1329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f478bfff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f478c09bea0 0x7f478c1329d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50900/0 (socket says 172.21.15.104:50900) 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f478b7fe640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f478c132f10 0x7f478c06d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:50274/0 (socket says 172.21.15.104:50274) 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f478bfff640 1 -- 172.21.15.104:0/3017832333 learned_addr learned my addr 172.21.15.104:0/3017832333 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f478bfff640 1 -- 172.21.15.104:0/3017832333 >> v1:172.21.15.104:6789/0 conn(0x7f478c132f10 legacy=0x7f478c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.594+0000 7f478bfff640 1 -- 172.21.15.104:0/3017832333 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c0706e0 con 0x7f478c09bea0 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f478bfff640 1 --2- 172.21.15.104:0/3017832333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c1329d0 crc :-1 s=READY pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e60be8ac991b79 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f478affd640 1 -- 172.21.15.104:0/3017832333 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f477c009e40 con 0x7f478c09bea0 2024-03-27T08:42:22.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f478affd640 1 -- 172.21.15.104:0/3017832333 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f477c00e230 con 0x7f478c09bea0 2024-03-27T08:42:22.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f478affd640 1 -- 172.21.15.104:0/3017832333 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f477c00e4d0 con 0x7f478c09bea0 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f4792a2d640 1 -- 172.21.15.104:0/3017832333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 msgr2=0x7f478c1329d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f4792a2d640 1 --2- 172.21.15.104:0/3017832333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c1329d0 crc :-1 s=READY pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f4792a2d640 1 -- 172.21.15.104:0/3017832333 shutdown_connections 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f4792a2d640 1 --2- 172.21.15.104:0/3017832333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c1329d0 unknown :-1 s=CLOSED pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.595+0000 7f4792a2d640 1 -- 172.21.15.104:0/3017832333 >> 172.21.15.104:0/3017832333 conn(0x7f478c0a1880 msgr2=0x7f478c0a1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.596+0000 7f4792a2d640 1 -- 172.21.15.104:0/3017832333 shutdown_connections 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.596+0000 7f4792a2d640 1 -- 172.21.15.104:0/3017832333 wait complete. 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.596+0000 7f4792a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:22.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.597+0000 7f4792a2d640 1 Processor -- start 2024-03-27T08:42:22.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.597+0000 7f4792a2d640 1 -- start start 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.598+0000 7f4792a2d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c11bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.598+0000 7f4792a2d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f478c11fcf0 con 0x7f478c09bea0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.598+0000 7f478bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c11bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.598+0000 7f478bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c11bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:50904/0 (socket says 172.21.15.104:50904) 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.598+0000 7f478bfff640 1 -- 172.21.15.104:0/3856543824 learned_addr learned my addr 172.21.15.104:0/3856543824 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f478bfff640 1 -- 172.21.15.104:0/3856543824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c12fd60 con 0x7f478c09bea0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f478bfff640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c11bc00 crc :-1 s=READY pgs=12900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f477c00ed90 con 0x7f478c09bea0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f477c015420 con 0x7f478c09bea0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c11e0a0 con 0x7f478c09bea0 2024-03-27T08:42:22.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f477c0156c0 con 0x7f478c09bea0 2024-03-27T08:42:22.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.599+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c11e8a0 con 0x7f478c09bea0 2024-03-27T08:42:22.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.600+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f477c015870 con 0x7f478c09bea0 2024-03-27T08:42:22.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.600+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f478c11e410 con 0x7f478c09bea0 2024-03-27T08:42:22.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.600+0000 7f4788ff9640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f47780387a0 0x7f477803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:22.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.600+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(691..691 src has 397..691) ==== 3791+0+0 (crc 0 0 0) 0x7f477c049a80 con 0x7f478c09bea0 2024-03-27T08:42:22.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.600+0000 7f478b7fe640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f47780387a0 0x7f477803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:22.594 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.601+0000 7f478b7fe640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f47780387a0 0x7f477803acb0 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:22.598 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.605+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f477c010c00 con 0x7f478c09bea0 2024-03-27T08:42:22.772 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:22.779+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "100"} v 0) -- 0x7f478c0693c0 con 0x7f478c09bea0 2024-03-27T08:42:23.033 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:23.040+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v692) ==== 221+0+0 (crc 0 0 0) 0x7f477c010820 con 0x7f478c09bea0 2024-03-27T08:42:23.137 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:23.144+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "100"} v 0) -- 0x7f478c0008d0 con 0x7f478c09bea0 2024-03-27T08:42:24.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.041+0000 7f4788ff9640 1 -- 172.21.15.104:0/3856543824 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v693) ==== 221+0+0 (crc 0 0 0) 0x7f477c010a00 con 0x7f478c09bea0 2024-03-27T08:42:24.036 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_bytes = 100 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f47780387a0 msgr2=0x7f477803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f47780387a0 0x7f477803acb0 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 msgr2=0x7f478c11bc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c11bc00 crc :-1 s=READY pgs=12900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 shutdown_connections 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f47780387a0 0x7f477803acb0 unknown :-1 s=CLOSED pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:24.037 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 --2- 172.21.15.104:0/3856543824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f478c09bea0 0x7f478c11bc00 unknown :-1 s=CLOSED pgs=12900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:24.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 >> 172.21.15.104:0/3856543824 conn(0x7f478c0a1880 msgr2=0x7f478c130f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:24.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.043+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 shutdown_connections 2024-03-27T08:42:24.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:24.044+0000 7f4792a2d640 1 -- 172.21.15.104:0/3856543824 wait complete. 2024-03-27T08:42:24.051 INFO:tasks.workunit.client.0.smithi104.stderr:+ sleep 30 2024-03-27T08:42:54.052 INFO:tasks.workunit.client.0.smithi104.stderr:+ pid=99706 2024-03-27T08:42:54.052 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_bytes 0 2024-03-27T08:42:54.052 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put two /etc/passwd 2024-03-27T08:42:54.096 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.102+0000 7f034dd83ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:54.101 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.108+0000 7f034dd83ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:54.101 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.108+0000 7f034dd83ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:54.102 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.109+0000 7f034dd83ec0 1 Processor -- start 2024-03-27T08:42:54.102 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.109+0000 7f034dd83ec0 1 -- start start 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034dd83ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034dd83ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x16260b0 con 0x16c0f70 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034679b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034679b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42950/0 (socket says 172.21.15.104:42950) 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034679b640 1 -- 172.21.15.104:0/4289853240 learned_addr learned my addr 172.21.15.104:0/4289853240 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034679b640 1 -- 172.21.15.104:0/4289853240 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x15c61e0 con 0x16c0f70 2024-03-27T08:42:54.104 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.110+0000 7f034679b640 1 --2- 172.21.15.104:0/4289853240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17d8f70 crc :-1 s=READY pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9649daff3e3465 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.105 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.111+0000 7f0344f98640 1 -- 172.21.15.104:0/4289853240 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f033400b5d0 con 0x16c0f70 2024-03-27T08:42:54.105 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.111+0000 7f0344f98640 1 -- 172.21.15.104:0/4289853240 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f033400b730 con 0x16c0f70 2024-03-27T08:42:54.105 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.111+0000 7f0344f98640 1 -- 172.21.15.104:0/4289853240 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f033400b9d0 con 0x16c0f70 2024-03-27T08:42:54.107 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 -- 172.21.15.104:0/4289853240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 msgr2=0x17d8f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:54.107 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/4289853240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17d8f70 crc :-1 s=READY pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:54.107 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 -- 172.21.15.104:0/4289853240 shutdown_connections 2024-03-27T08:42:54.107 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/4289853240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17d8f70 unknown :-1 s=CLOSED pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:54.107 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 -- 172.21.15.104:0/4289853240 >> 172.21.15.104:0/4289853240 conn(0x15daea0 msgr2=0x17dc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:54.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 -- 172.21.15.104:0/4289853240 shutdown_connections 2024-03-27T08:42:54.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.112+0000 7f034dd83ec0 1 -- 172.21.15.104:0/4289853240 wait complete. 2024-03-27T08:42:54.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.113+0000 7f034dd83ec0 1 Processor -- start 2024-03-27T08:42:54.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.113+0000 7f034dd83ec0 1 -- start start 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.114+0000 7f034dd83ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.114+0000 7f034dd83ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1623b30 con 0x16c0f70 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.114+0000 7f034679b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.114+0000 7f034679b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17e8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42962/0 (socket says 172.21.15.104:42962) 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.114+0000 7f034679b640 1 -- 172.21.15.104:0/1052099588 learned_addr learned my addr 172.21.15.104:0/1052099588 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.114+0000 7f034679b640 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x15b8e30 con 0x16c0f70 2024-03-27T08:42:54.113 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.115+0000 7f034679b640 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17e8c30 crc :-1 s=READY pgs=12904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.115+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0334012910 con 0x16c0f70 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.115+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x15b8890 con 0x16c0f70 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.115+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0334005040 con 0x16c0f70 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.115+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0334012ba0 con 0x16c0f70 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.115+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x17c5f40 con 0x16c0f70 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.116+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f033400d040 con 0x16c0f70 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.116+0000 7f033effd640 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f032c036fe0 0x7f032c0394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.116+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(694..694 src has 397..694) ==== 3791+0+0 (crc 0 0 0) 0x7f033400a040 con 0x16c0f70 2024-03-27T08:42:54.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.116+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=695}) -- 0x7f032c03d1f0 con 0x16c0f70 2024-03-27T08:42:54.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.116+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> v2:172.21.15.104:6801/1224298329 conn(0x16c1a90 0x17c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.117+0000 7f0345f9a640 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f032c036fe0 0x7f032c0394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:54.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.117+0000 7f0345f9a640 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f032c036fe0 0x7f032c0394f0 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.118+0000 7f0346f9c640 1 --2- 172.21.15.104:0/1052099588 >> v2:172.21.15.104:6801/1224298329 conn(0x16c1a90 0x17c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:42:54.116 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.122+0000 7f0346f9c640 1 --2- 172.21.15.104:0/1052099588 >> v2:172.21.15.104:6801/1224298329 conn(0x16c1a90 0x17c31d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.154 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.152+0000 7f6c3242d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:54.155 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.162+0000 7f6c3242d640 1 Processor -- start 2024-03-27T08:42:54.155 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.162+0000 7f6c3242d640 1 -- start start 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c3242d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c2c137a80 0x7f6c2c139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c3242d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2c06d980 con 0x7f6c2c09f860 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c3242d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2c06dc20 con 0x7f6c2c137a80 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2b7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c2c137a80 0x7f6c2c139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2bfff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f6c2c09f860 0x7f6c2c09fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:60698/0 (socket says 172.21.15.104:60698) 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2b7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6c2c137a80 0x7f6c2c139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42972/0 (socket says 172.21.15.104:42972) 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2bfff640 1 -- 172.21.15.104:0/2936028238 learned_addr learned my addr 172.21.15.104:0/2936028238 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2b7fe640 1 -- 172.21.15.104:0/2936028238 >> v1:172.21.15.104:6789/0 conn(0x7f6c2c09f860 legacy=0x7f6c2c09fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2b7fe640 1 -- 172.21.15.104:0/2936028238 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c09f430 con 0x7f6c2c137a80 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2b7fe640 1 --2- 172.21.15.104:0/2936028238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c137a80 0x7f6c2c139ea0 crc :-1 s=READY pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc79c31f4a2775b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.157 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.163+0000 7f6c2affd640 1 -- 172.21.15.104:0/2936028238 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6c1c009d90 con 0x7f6c2c137a80 2024-03-27T08:42:54.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c2affd640 1 -- 172.21.15.104:0/2936028238 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6c1c00e230 con 0x7f6c2c137a80 2024-03-27T08:42:54.158 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c2affd640 1 -- 172.21.15.104:0/2936028238 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6c1c00e4d0 con 0x7f6c2c137a80 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 -- 172.21.15.104:0/2936028238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c137a80 msgr2=0x7f6c2c139ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 --2- 172.21.15.104:0/2936028238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c137a80 0x7f6c2c139ea0 crc :-1 s=READY pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 -- 172.21.15.104:0/2936028238 shutdown_connections 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 --2- 172.21.15.104:0/2936028238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c137a80 0x7f6c2c139ea0 unknown :-1 s=CLOSED pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 -- 172.21.15.104:0/2936028238 >> 172.21.15.104:0/2936028238 conn(0x7f6c2c0a4bc0 msgr2=0x7f6c2c0a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 -- 172.21.15.104:0/2936028238 shutdown_connections 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.164+0000 7f6c3242d640 1 -- 172.21.15.104:0/2936028238 wait complete. 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.165+0000 7f6c3242d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:54.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.166+0000 7f6c3242d640 1 Processor -- start 2024-03-27T08:42:54.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.166+0000 7f6c3242d640 1 -- start start 2024-03-27T08:42:54.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.167+0000 7f6c3242d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 0x7f6c2c11ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.167+0000 7f6c3242d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c121440 con 0x7f6c2c09f860 2024-03-27T08:42:54.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.167+0000 7f6c2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 0x7f6c2c11ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:54.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.167+0000 7f6c2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 0x7f6c2c11ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42984/0 (socket says 172.21.15.104:42984) 2024-03-27T08:42:54.161 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.167+0000 7f6c2bfff640 1 -- 172.21.15.104:0/1032702846 learned_addr learned my addr 172.21.15.104:0/1032702846 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.167+0000 7f6c2bfff640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c122ef0 con 0x7f6c2c09f860 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.168+0000 7f6c2bfff640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 0x7f6c2c11ee90 crc :-1 s=READY pgs=12907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.168+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6c14005a60 con 0x7f6c2c09f860 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.168+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6c14005bc0 con 0x7f6c2c09f860 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.168+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c120020 con 0x7f6c2c09f860 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.168+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6c14005e60 con 0x7f6c2c09f860 2024-03-27T08:42:54.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.168+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c122480 con 0x7f6c2c09f860 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.169+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f6c140070b0 con 0x7f6c2c09f860 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.169+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c122ce0 con 0x7f6c2c09f860 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.169+0000 7f6c28ff9640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6c0c0387a0 0x7f6c0c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.169+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(694..694 src has 397..694) ==== 3791+0+0 (crc 0 0 0) 0x7f6c14048330 con 0x7f6c2c09f860 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.169+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=695}) -- 0x7f6c0c03ea10 con 0x7f6c2c09f860 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.169+0000 7f6c2b7fe640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6c0c0387a0 0x7f6c0c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:54.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.170+0000 7f6c2b7fe640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6c0c0387a0 0x7f6c0c03acb0 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:54.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.174+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f6c1400d720 con 0x7f6c2c09f860 2024-03-27T08:42:54.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:54.350+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"} v 0) -- 0x7f6c2c0693c0 con 0x7f6c2c09f860 2024-03-27T08:42:55.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:55.024+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v695) ==== 217+0+0 (crc 0 0 0) 0x7f6c1400d150 con 0x7f6c2c09f860 2024-03-27T08:42:55.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:55.024+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(695..695 src has 397..695) ==== 628+0+0 (crc 0 0 0) 0x7f6c1400d510 con 0x7f6c2c09f860 2024-03-27T08:42:55.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:55.024+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 6 ==== osd_map(695..695 src has 397..695) ==== 628+0+0 (crc 0 0 0) 0x7f033400b050 con 0x16c0f70 2024-03-27T08:42:55.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:55.024+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=696}) -- 0x7f6c0c03f950 con 0x7f6c2c09f860 2024-03-27T08:42:55.018 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:55.024+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=696}) -- 0x7f032c03d880 con 0x16c0f70 2024-03-27T08:42:55.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:55.166+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"} v 0) -- 0x7f6c2c120210 con 0x7f6c2c09f860 2024-03-27T08:42:56.021 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.028+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v696) ==== 217+0+0 (crc 0 0 0) 0x7f6c1400d330 con 0x7f6c2c09f860 2024-03-27T08:42:56.022 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_bytes = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:42:56.024 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6c0c0387a0 msgr2=0x7f6c0c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:56.024 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6c0c0387a0 0x7f6c0c03acb0 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:56.024 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c28ff9640 1 -- 172.21.15.104:0/1032702846 <== mon.0 v2:172.21.15.104:3300/0 10 ==== osd_map(696..696 src has 397..696) ==== 628+0+0 (crc 0 0 0) 0x7f6c140090b0 con 0x7f6c2c09f860 2024-03-27T08:42:56.024 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 msgr2=0x7f6c2c11ee90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:56.024 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 0x7f6c2c11ee90 crc :-1 s=READY pgs=12907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 7 ==== osd_map(696..696 src has 397..696) ==== 628+0+0 (crc 0 0 0) 0x7f0334004e30 con 0x16c0f70 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 shutdown_connections 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f6c0c0387a0 0x7f6c0c03acb0 unknown :-1 s=CLOSED pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 --2- 172.21.15.104:0/1032702846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c09f860 0x7f6c2c11ee90 unknown :-1 s=CLOSED pgs=12907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 >> 172.21.15.104:0/1032702846 conn(0x7f6c2c0a4bc0 msgr2=0x7f6c2c134680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=697}) -- 0x7f032c03daa0 con 0x16c0f70 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 shutdown_connections 2024-03-27T08:42:56.025 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.030+0000 7f6c3242d640 1 -- 172.21.15.104:0/1032702846 wait complete. 2024-03-27T08:42:56.034 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_objects 0 2024-03-27T08:42:56.127 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.126+0000 7fb3bba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:56.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.135+0000 7fb3bba6d640 1 Processor -- start 2024-03-27T08:42:56.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.135+0000 7fb3bba6d640 1 -- start start 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3bba6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fb3b409fd90 0x7fb3b4134d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3bba6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b40a01a0 con 0x7fb3b409f980 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3bba6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b406ea70 con 0x7fb3b409fd90 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3b9809640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fb3b409f980 0x7fb3b4134680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33092/0 (socket says 172.21.15.104:33092) 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3b9809640 1 -- 172.21.15.104:0/1284788434 learned_addr learned my addr 172.21.15.104:0/1284788434 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3b9008640 1 --2- 172.21.15.104:0/1284788434 >> v2:172.21.15.104:3300/0 conn(0x7fb3b409fd90 0x7fb3b4134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:56.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3b9008640 1 -- 172.21.15.104:0/1284788434 >> v1:172.21.15.104:6789/0 conn(0x7fb3b409f980 legacy=0x7fb3b4134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:56.131 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.136+0000 7fb3b9008640 1 -- 172.21.15.104:0/1284788434 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b409f2e0 con 0x7fb3b409fd90 2024-03-27T08:42:56.131 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3b9008640 1 --2- 172.21.15.104:0/1284788434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409fd90 0x7fb3b4134d90 crc :-1 s=READY pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a578b444994671a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:56.131 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3b8807640 1 -- 172.21.15.104:0/1284788434 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3a4009d90 con 0x7fb3b409fd90 2024-03-27T08:42:56.131 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3b8807640 1 -- 172.21.15.104:0/1284788434 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb3a400e230 con 0x7fb3b409fd90 2024-03-27T08:42:56.131 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3b8807640 1 -- 172.21.15.104:0/1284788434 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3a400e4d0 con 0x7fb3b409fd90 2024-03-27T08:42:56.132 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3bba6d640 1 -- 172.21.15.104:0/1284788434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409fd90 msgr2=0x7fb3b4134d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:56.132 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3bba6d640 1 --2- 172.21.15.104:0/1284788434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409fd90 0x7fb3b4134d90 crc :-1 s=READY pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:56.132 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.137+0000 7fb3bba6d640 1 -- 172.21.15.104:0/1284788434 shutdown_connections 2024-03-27T08:42:56.132 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.138+0000 7fb3bba6d640 1 --2- 172.21.15.104:0/1284788434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409fd90 0x7fb3b4134d90 unknown :-1 s=CLOSED pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:56.132 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.138+0000 7fb3bba6d640 1 -- 172.21.15.104:0/1284788434 >> 172.21.15.104:0/1284788434 conn(0x7fb3b40a4bc0 msgr2=0x7fb3b40a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:56.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.138+0000 7fb3bba6d640 1 -- 172.21.15.104:0/1284788434 shutdown_connections 2024-03-27T08:42:56.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.138+0000 7fb3bba6d640 1 -- 172.21.15.104:0/1284788434 wait complete. 2024-03-27T08:42:56.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.138+0000 7fb3bba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:56.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.139+0000 7fb3bba6d640 1 Processor -- start 2024-03-27T08:42:56.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.139+0000 7fb3bba6d640 1 -- start start 2024-03-27T08:42:56.134 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.140+0000 7fb3bba6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 0x7fb3b411e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:56.134 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.140+0000 7fb3bba6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41220c0 con 0x7fb3b409f980 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.140+0000 7fb3b9809640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 0x7fb3b411e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.140+0000 7fb3b9809640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 0x7fb3b411e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51826/0 (socket says 172.21.15.104:51826) 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.140+0000 7fb3b9809640 1 -- 172.21.15.104:0/92310430 learned_addr learned my addr 172.21.15.104:0/92310430 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.140+0000 7fb3b9809640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b4132250 con 0x7fb3b409f980 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.141+0000 7fb3b9809640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 0x7fb3b411e000 crc :-1 s=READY pgs=12910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.141+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3b0005a60 con 0x7fb3b409f980 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.141+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb3b0005bc0 con 0x7fb3b409f980 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.141+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3b0005e60 con 0x7fb3b409f980 2024-03-27T08:42:56.135 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.141+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b4069180 con 0x7fb3b409f980 2024-03-27T08:42:56.136 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.141+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b4131f80 con 0x7fb3b409f980 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.142+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fb3b00070b0 con 0x7fb3b409f980 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.142+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b41203d0 con 0x7fb3b409f980 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.142+0000 7fb3aa7fc640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb3980387a0 0x7fb39803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.142+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(696..696 src has 397..696) ==== 3791+0+0 (crc 0 0 0) 0x7fb3b0048170 con 0x7fb3b409f980 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.142+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=697}) -- 0x7fb39803e9b0 con 0x7fb3b409f980 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.142+0000 7fb3b9008640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb3980387a0 0x7fb39803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:56.138 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.143+0000 7fb3b9008640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb3980387a0 0x7fb39803acb0 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:56.140 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.147+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fb3b000c700 con 0x7fb3b409f980 2024-03-27T08:42:56.316 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:56.321+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"} v 0) -- 0x7fb3b4069180 con 0x7fb3b409f980 2024-03-27T08:42:57.007 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.014+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v697) ==== 221+0+0 (crc 0 0 0) 0x7fb3b4069180 con 0x7fb3b409f980 2024-03-27T08:42:57.008 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.014+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(697..697 src has 397..697) ==== 628+0+0 (crc 0 0 0) 0x7fb3b000e9d0 con 0x7fb3b409f980 2024-03-27T08:42:57.008 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.014+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(697..697 src has 397..697) ==== 628+0+0 (crc 0 0 0) 0x7f033400c740 con 0x16c0f70 2024-03-27T08:42:57.008 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.014+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=698}) -- 0x7fb39803f960 con 0x7fb3b409f980 2024-03-27T08:42:57.008 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.014+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=698}) -- 0x7f032c03dc90 con 0x16c0f70 2024-03-27T08:42:57.008 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.014+0000 7f033effd640 1 -- 172.21.15.104:0/1052099588 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 200.6 200:6b5d76a5:::two:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e697) -- 0x7f032c03de80 con 0x16c1a90 2024-03-27T08:42:57.150 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.157+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"} v 0) -- 0x7fb3b4069180 con 0x7fb3b409f980 2024-03-27T08:42:57.269 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.275+0000 7f0346f9c640 1 -- 172.21.15.104:0/1052099588 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 two [writefull 0~11812] v697'2 uv2 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7f033800a430 con 0x16c1a90 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.275+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 >> v2:172.21.15.104:6801/1224298329 conn(0x16c1a90 msgr2=0x17c31d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.275+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> v2:172.21.15.104:6801/1224298329 conn(0x16c1a90 0x17c31d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f032c036fe0 msgr2=0x7f032c0394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f032c036fe0 0x7f032c0394f0 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 msgr2=0x17e8c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17e8c30 crc :-1 s=READY pgs=12904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 shutdown_connections 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f032c036fe0 0x7f032c0394f0 unknown :-1 s=CLOSED pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> v2:172.21.15.104:6801/1224298329 conn(0x16c1a90 0x17c31d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 --2- 172.21.15.104:0/1052099588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x16c0f70 0x17e8c30 unknown :-1 s=CLOSED pgs=12904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:57.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 >> 172.21.15.104:0/1052099588 conn(0x15daea0 msgr2=0x17da790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:57.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 shutdown_connections 2024-03-27T08:42:57.271 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:57.276+0000 7f034dd83ec0 1 -- 172.21.15.104:0/1052099588 wait complete. 2024-03-27T08:42:58.026 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.032+0000 7fb3aa7fc640 1 -- 172.21.15.104:0/92310430 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v698) ==== 221+0+0 (crc 0 0 0) 0x7fb3b4069180 con 0x7fb3b409f980 2024-03-27T08:42:58.027 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_objects = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:42:58.028 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb3980387a0 msgr2=0x7fb39803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.028 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb3980387a0 0x7fb39803acb0 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.028 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 msgr2=0x7fb3b411e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.028 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 0x7fb3b411e000 crc :-1 s=READY pgs=12910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.028 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 shutdown_connections 2024-03-27T08:42:58.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fb3980387a0 0x7fb39803acb0 unknown :-1 s=CLOSED pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 --2- 172.21.15.104:0/92310430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3b409f980 0x7fb3b411e000 unknown :-1 s=CLOSED pgs=12910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 >> 172.21.15.104:0/92310430 conn(0x7fb3b40a4bc0 msgr2=0x7fb3b40a16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:58.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 shutdown_connections 2024-03-27T08:42:58.029 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.035+0000 7fb3bba6d640 1 -- 172.21.15.104:0/92310430 wait complete. 2024-03-27T08:42:58.038 INFO:tasks.workunit.client.0.smithi104.stderr:+ wait 99706 2024-03-27T08:42:58.039 INFO:tasks.workunit.client.0.smithi104.stderr:+ '[' 0 -ne 0 ']' 2024-03-27T08:42:58.039 INFO:tasks.workunit.client.0.smithi104.stderr:+ true 2024-03-27T08:42:58.039 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put three /etc/passwd 2024-03-27T08:42:58.076 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.083+0000 7f07f4adbec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:58.083 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.090+0000 7f07f4adbec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:58.084 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.090+0000 7f07f4adbec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:58.084 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.091+0000 7f07f4adbec0 1 Processor -- start 2024-03-27T08:42:58.084 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.091+0000 7f07f4adbec0 1 -- start start 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.092+0000 7f07f4adbec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1bc61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.092+0000 7f07f4adbec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1b310b0 con 0x1bb18f0 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.092+0000 7f07e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1bc61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.092+0000 7f07e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1bc61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51852/0 (socket says 172.21.15.104:51852) 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.092+0000 7f07e7fff640 1 -- 172.21.15.104:0/1008106825 learned_addr learned my addr 172.21.15.104:0/1008106825 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.092+0000 7f07e7fff640 1 -- 172.21.15.104:0/1008106825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1ad11e0 con 0x1bb18f0 2024-03-27T08:42:58.086 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.093+0000 7f07e7fff640 1 --2- 172.21.15.104:0/1008106825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1bc61a0 crc :-1 s=READY pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fd0a6d197e857a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.087 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.093+0000 7f07e67fc640 1 -- 172.21.15.104:0/1008106825 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07d8029c00 con 0x1bb18f0 2024-03-27T08:42:58.087 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.093+0000 7f07e67fc640 1 -- 172.21.15.104:0/1008106825 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f07d800a040 con 0x1bb18f0 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.093+0000 7f07e67fc640 1 -- 172.21.15.104:0/1008106825 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07d800a2e0 con 0x1bb18f0 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1008106825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 msgr2=0x1bc61a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1008106825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1bc61a0 crc :-1 s=READY pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1008106825 shutdown_connections 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1008106825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1bc61a0 unknown :-1 s=CLOSED pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1008106825 >> 172.21.15.104:0/1008106825 conn(0x1ae5ea0 msgr2=0x1c5f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1008106825 shutdown_connections 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.094+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1008106825 wait complete. 2024-03-27T08:42:58.088 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.095+0000 7f07f4adbec0 1 Processor -- start 2024-03-27T08:42:58.089 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.095+0000 7f07f4adbec0 1 -- start start 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.096+0000 7f07f4adbec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1c5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.096+0000 7f07f4adbec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1b2eb30 con 0x1bb18f0 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.096+0000 7f07e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1c5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.096+0000 7f07e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1c5a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51864/0 (socket says 172.21.15.104:51864) 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.096+0000 7f07e7fff640 1 -- 172.21.15.104:0/1192995158 learned_addr learned my addr 172.21.15.104:0/1192995158 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.096+0000 7f07e7fff640 1 -- 172.21.15.104:0/1192995158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1ac3e30 con 0x1bb18f0 2024-03-27T08:42:58.090 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.097+0000 7f07e7fff640 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1c5a8e0 crc :-1 s=READY pgs=12914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.097+0000 7f07e57fa640 1 -- 172.21.15.104:0/1192995158 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07d803b040 con 0x1bb18f0 2024-03-27T08:42:58.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.097+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1ac3890 con 0x1bb18f0 2024-03-27T08:42:58.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.097+0000 7f07e57fa640 1 -- 172.21.15.104:0/1192995158 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f07d8006c60 con 0x1bb18f0 2024-03-27T08:42:58.091 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.097+0000 7f07e57fa640 1 -- 172.21.15.104:0/1192995158 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07d8029c00 con 0x1bb18f0 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.097+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1c62c20 con 0x1bb18f0 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.098+0000 7f07e57fa640 1 -- 172.21.15.104:0/1192995158 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f07d8035450 con 0x1bb18f0 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.098+0000 7f07e57fa640 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f07d4036fe0 0x7f07d40394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.098+0000 7f07e57fa640 1 -- 172.21.15.104:0/1192995158 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(698..698 src has 397..698) ==== 3791+0+0 (crc 0 0 0) 0x7f07d802e040 con 0x1bb18f0 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.098+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> v2:172.21.15.177:6807/929709834 conn(0x1bb2240 0x1cf6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.098+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 200.3 200:c57798d7:::three:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e698) -- 0x1c483d0 con 0x1bb2240 2024-03-27T08:42:58.092 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.099+0000 7f07e77fe640 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f07d4036fe0 0x7f07d40394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:58.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.099+0000 7f07e77fe640 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f07d4036fe0 0x7f07d40394f0 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.093 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.099+0000 7f07ecf9b640 1 --2- 172.21.15.104:0/1192995158 >> v2:172.21.15.177:6807/929709834 conn(0x1bb2240 0x1cf6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:42:58.095 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.102+0000 7f07ecf9b640 1 --2- 172.21.15.104:0/1192995158 >> v2:172.21.15.177:6807/929709834 conn(0x1bb2240 0x1cf6340 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07ecf9b640 1 -- 172.21.15.104:0/1192995158 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 three [writefull 0~11812] v698'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f07e000a430 con 0x1bb2240 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 >> v2:172.21.15.177:6807/929709834 conn(0x1bb2240 msgr2=0x1cf6340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> v2:172.21.15.177:6807/929709834 conn(0x1bb2240 0x1cf6340 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f07d4036fe0 msgr2=0x7f07d40394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f07d4036fe0 0x7f07d40394f0 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 msgr2=0x1c5a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1c5a8e0 crc :-1 s=READY pgs=12914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 shutdown_connections 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> v2:172.21.15.177:6807/929709834 conn(0x1bb2240 0x1cf6340 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.281 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f07d4036fe0 0x7f07d40394f0 unknown :-1 s=CLOSED pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 --2- 172.21.15.104:0/1192995158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1bb18f0 0x1c5a8e0 unknown :-1 s=CLOSED pgs=12914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.286+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 >> 172.21.15.104:0/1192995158 conn(0x1ae5ea0 msgr2=0x1c5ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:58.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.287+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 shutdown_connections 2024-03-27T08:42:58.282 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.287+0000 7f07f4adbec0 1 -- 172.21.15.104:0/1192995158 wait complete. 2024-03-27T08:42:58.282 INFO:tasks.workunit.client.0.smithi104.stderr:++ uuidgen 2024-03-27T08:42:58.283 INFO:tasks.workunit.client.0.smithi104.stderr:+ pp=b94fed44-a57c-48b9-b4e4-7a1f5935fb7f 2024-03-27T08:42:58.283 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool create b94fed44-a57c-48b9-b4e4-7a1f5935fb7f 12 2024-03-27T08:42:58.375 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.374+0000 7f3b1aa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:58.375 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.382+0000 7f3b1aa6d640 1 Processor -- start 2024-03-27T08:42:58.375 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.382+0000 7f3b1aa6d640 1 -- start start 2024-03-27T08:42:58.378 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.383+0000 7f3b1aa6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3b1409fee0 0x7f3b1406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.378 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.383+0000 7f3b1aa6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b1409fd00 con 0x7f3b1406d780 2024-03-27T08:42:58.378 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.383+0000 7f3b1aa6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f3b1406e720 con 0x7f3b1409fee0 2024-03-27T08:42:58.378 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.383+0000 7f3b18809640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3b1409fee0 0x7f3b1406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.383+0000 7f3b18809640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3b1409fee0 0x7f3b1406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51872/0 (socket says 172.21.15.104:51872) 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.383+0000 7f3b18809640 1 -- 172.21.15.104:0/2947213000 learned_addr learned my addr 172.21.15.104:0/2947213000 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b18809640 1 -- 172.21.15.104:0/2947213000 >> v1:172.21.15.104:6789/0 conn(0x7f3b1406d780 legacy=0x7f3b14130fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b18809640 1 -- 172.21.15.104:0/2947213000 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1409f9f0 con 0x7f3b1409fee0 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b137fe640 1 -- 172.21.15.104:0/2947213000 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3680375576 0 0) 0x7f3b1409fd00 con 0x7f3b1406d780 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b18809640 1 --2- 172.21.15.104:0/2947213000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1409fee0 0x7f3b1406d240 crc :-1 s=READY pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b9dd2385e36b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b137fe640 1 -- 172.21.15.104:0/2947213000 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3b08009e50 con 0x7f3b1409fee0 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b137fe640 1 -- 172.21.15.104:0/2947213000 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3b0800e230 con 0x7f3b1409fee0 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.384+0000 7f3b137fe640 1 -- 172.21.15.104:0/2947213000 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3b0800e4d0 con 0x7f3b1409fee0 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2947213000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1409fee0 msgr2=0x7f3b1406d240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 --2- 172.21.15.104:0/2947213000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1409fee0 0x7f3b1406d240 crc :-1 s=READY pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2947213000 shutdown_connections 2024-03-27T08:42:58.379 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 --2- 172.21.15.104:0/2947213000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1409fee0 0x7f3b1406d240 unknown :-1 s=CLOSED pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:58.380 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2947213000 >> 172.21.15.104:0/2947213000 conn(0x7f3b1409aa00 msgr2=0x7f3b1409d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:58.380 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2947213000 shutdown_connections 2024-03-27T08:42:58.380 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2947213000 wait complete. 2024-03-27T08:42:58.380 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.385+0000 7f3b1aa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:58.381 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.386+0000 7f3b1aa6d640 1 Processor -- start 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.386+0000 7f3b1aa6d640 1 -- start start 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.387+0000 7f3b1aa6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 0x7f3b141a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.387+0000 7f3b1aa6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b141a88d0 con 0x7f3b1406d780 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b18809640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 0x7f3b141a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b18809640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 0x7f3b141a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51874/0 (socket says 172.21.15.104:51874) 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b18809640 1 -- 172.21.15.104:0/2904912530 learned_addr learned my addr 172.21.15.104:0/2904912530 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b18809640 1 -- 172.21.15.104:0/2904912530 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b141a6c20 con 0x7f3b1406d780 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b18809640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 0x7f3b141a4710 crc :-1 s=READY pgs=12917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3b0800edc0 con 0x7f3b1406d780 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3b08015420 con 0x7f3b1406d780 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b141b8b10 con 0x7f3b1406d780 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3b080156c0 con 0x7f3b1406d780 2024-03-27T08:42:58.382 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.388+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b141a7be0 con 0x7f3b1406d780 2024-03-27T08:42:58.383 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.389+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f3b08015870 con 0x7f3b1406d780 2024-03-27T08:42:58.383 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.389+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b141a84f0 con 0x7f3b1406d780 2024-03-27T08:42:58.384 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.389+0000 7f3b117fa640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3af80387a0 0x7f3af803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:58.384 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.389+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(698..698 src has 397..698) ==== 3791+0+0 (crc 0 0 0) 0x7f3b08049bb0 con 0x7f3b1406d780 2024-03-27T08:42:58.384 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.389+0000 7f3b13fff640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3af80387a0 0x7f3af803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:58.384 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.390+0000 7f3b13fff640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3af80387a0 0x7f3af803acb0 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:58.388 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.395+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f3b08010be0 con 0x7f3b1406d780 2024-03-27T08:42:58.563 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:58.569+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pg_num": 12} v 0) -- 0x7f3b140693c0 con 0x7f3b1406d780 2024-03-27T08:42:59.039 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.045+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pg_num": 12}]=0 pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' created v699) ==== 176+0+0 (crc 0 0 0) 0x7f3b080107d0 con 0x7f3b1406d780 2024-03-27T08:42:59.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.194+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pg_num": 12} v 0) -- 0x7f3b141a8090 con 0x7f3b1406d780 2024-03-27T08:42:59.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.195+0000 7f3b117fa640 1 -- 172.21.15.104:0/2904912530 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pg_num": 12}]=0 pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' already exists v699) ==== 183+0+0 (crc 0 0 0) 0x7f3b080109b0 con 0x7f3b1406d780 2024-03-27T08:42:59.188 INFO:tasks.workunit.client.0.smithi104.stderr:pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' already exists 2024-03-27T08:42:59.192 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.198+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3af80387a0 msgr2=0x7f3af803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:59.192 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.198+0000 7f3b1aa6d640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3af80387a0 0x7f3af803acb0 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:59.192 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.198+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 msgr2=0x7f3b141a4710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.198+0000 7f3b1aa6d640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 0x7f3b141a4710 crc :-1 s=READY pgs=12917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.199+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 shutdown_connections 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.199+0000 7f3b1aa6d640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3af80387a0 0x7f3af803acb0 unknown :-1 s=CLOSED pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.199+0000 7f3b1aa6d640 1 --2- 172.21.15.104:0/2904912530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b1406d780 0x7f3b141a4710 unknown :-1 s=CLOSED pgs=12917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.199+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 >> 172.21.15.104:0/2904912530 conn(0x7f3b1409aa00 msgr2=0x7f3b1419db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.199+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 shutdown_connections 2024-03-27T08:42:59.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.199+0000 7f3b1aa6d640 1 -- 172.21.15.104:0/2904912530 wait complete. 2024-03-27T08:42:59.208 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool application enable b94fed44-a57c-48b9-b4e4-7a1f5935fb7f rados 2024-03-27T08:42:59.339 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.329+0000 7f325ef9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:59.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.346+0000 7f325ef9d640 1 Processor -- start 2024-03-27T08:42:59.340 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.347+0000 7f325ef9d640 1 -- start start 2024-03-27T08:42:59.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f325ef9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f325809f6e0 0x7f325809faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:59.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f325ef9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f325806eb50 con 0x7f3258134800 2024-03-27T08:42:59.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f325ef9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f325809f2f0 con 0x7f325809f6e0 2024-03-27T08:42:59.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f325cd39640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f325809f6e0 0x7f325809faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:59.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f3257fff640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f3258134800 0x7f3258134c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33122/0 (socket says 172.21.15.104:33122) 2024-03-27T08:42:59.342 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f3257fff640 1 -- 172.21.15.104:0/3114731756 learned_addr learned my addr 172.21.15.104:0/3114731756 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:42:59.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f325cd39640 1 -- 172.21.15.104:0/3114731756 >> v1:172.21.15.104:6789/0 conn(0x7f3258134800 legacy=0x7f3258134c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:59.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.348+0000 7f325cd39640 1 -- 172.21.15.104:0/3114731756 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32580a0b60 con 0x7f325809f6e0 2024-03-27T08:42:59.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.349+0000 7f325cd39640 1 --2- 172.21.15.104:0/3114731756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325809faf0 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efe4b7d9624b476 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:59.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.349+0000 7f32577fe640 1 -- 172.21.15.104:0/3114731756 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f324c009d90 con 0x7f325809f6e0 2024-03-27T08:42:59.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.349+0000 7f32577fe640 1 -- 172.21.15.104:0/3114731756 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f324c00e230 con 0x7f325809f6e0 2024-03-27T08:42:59.343 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.349+0000 7f32577fe640 1 -- 172.21.15.104:0/3114731756 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f324c00e4d0 con 0x7f325809f6e0 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 -- 172.21.15.104:0/3114731756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 msgr2=0x7f325809faf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 --2- 172.21.15.104:0/3114731756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325809faf0 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 -- 172.21.15.104:0/3114731756 shutdown_connections 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 --2- 172.21.15.104:0/3114731756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325809faf0 unknown :-1 s=CLOSED pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 -- 172.21.15.104:0/3114731756 >> 172.21.15.104:0/3114731756 conn(0x7f32580a1860 msgr2=0x7f32580a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 -- 172.21.15.104:0/3114731756 shutdown_connections 2024-03-27T08:42:59.344 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.350+0000 7f325ef9d640 1 -- 172.21.15.104:0/3114731756 wait complete. 2024-03-27T08:42:59.345 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.351+0000 7f325ef9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:59.346 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.353+0000 7f325ef9d640 1 Processor -- start 2024-03-27T08:42:59.346 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.353+0000 7f325ef9d640 1 -- start start 2024-03-27T08:42:59.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.354+0000 7f325ef9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325811dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:59.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.354+0000 7f325ef9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3258122040 con 0x7f325809f6e0 2024-03-27T08:42:59.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f325cd39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325811dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:59.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f325cd39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325811dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51898/0 (socket says 172.21.15.104:51898) 2024-03-27T08:42:59.348 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f325cd39640 1 -- 172.21.15.104:0/2192710367 learned_addr learned my addr 172.21.15.104:0/2192710367 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:42:59.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f325cd39640 1 -- 172.21.15.104:0/2192710367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3258121a10 con 0x7f325809f6e0 2024-03-27T08:42:59.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f325cd39640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325811dee0 crc :-1 s=READY pgs=12920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:59.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f324c014620 con 0x7f325809f6e0 2024-03-27T08:42:59.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f324c014780 con 0x7f325809f6e0 2024-03-27T08:42:59.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.355+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3258120330 con 0x7f325809f6e0 2024-03-27T08:42:59.349 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.356+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3258132200 con 0x7f325809f6e0 2024-03-27T08:42:59.350 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.356+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f324c014a20 con 0x7f325809f6e0 2024-03-27T08:42:59.350 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.357+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f324c014bd0 con 0x7f325809f6e0 2024-03-27T08:42:59.350 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.357+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3258120780 con 0x7f325809f6e0 2024-03-27T08:42:59.350 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.357+0000 7f32557fa640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f323c0387a0 0x7f323c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:59.351 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.357+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(699..699 src has 397..699) ==== 4166+0+0 (crc 0 0 0) 0x7f324c019040 con 0x7f325809f6e0 2024-03-27T08:42:59.351 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.357+0000 7f3257fff640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f323c0387a0 0x7f323c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:59.351 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.358+0000 7f3257fff640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f323c0387a0 0x7f323c03acb0 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:59.361 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.368+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f324c07b7c0 con 0x7f325809f6e0 2024-03-27T08:42:59.568 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:42:59.575+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "app": "rados"} v 0) -- 0x7f325811f3e0 con 0x7f325809f6e0 2024-03-27T08:43:00.043 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:00.050+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "app": "rados"}]=0 enabled application 'rados' on pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' v700) ==== 213+0+0 (crc 0 0 0) 0x7f324c018190 con 0x7f325809f6e0 2024-03-27T08:43:00.227 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:00.233+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "app": "rados"} v 0) -- 0x7f32580693c0 con 0x7f325809f6e0 2024-03-27T08:43:01.050 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.057+0000 7f32557fa640 1 -- 172.21.15.104:0/2192710367 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "app": "rados"}]=0 enabled application 'rados' on pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' v701) ==== 213+0+0 (crc 0 0 0) 0x7f324c00fb30 con 0x7f325809f6e0 2024-03-27T08:43:01.051 INFO:tasks.workunit.client.0.smithi104.stderr:enabled application 'rados' on pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' 2024-03-27T08:43:01.052 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f323c0387a0 msgr2=0x7f323c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f323c0387a0 0x7f323c03acb0 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 msgr2=0x7f325811dee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325811dee0 crc :-1 s=READY pgs=12920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 shutdown_connections 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f323c0387a0 0x7f323c03acb0 unknown :-1 s=CLOSED pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 --2- 172.21.15.104:0/2192710367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f325809f6e0 0x7f325811dee0 unknown :-1 s=CLOSED pgs=12920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 >> 172.21.15.104:0/2192710367 conn(0x7f32580a1860 msgr2=0x7f32581bc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 shutdown_connections 2024-03-27T08:43:01.053 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.059+0000 7f325ef9d640 1 -- 172.21.15.104:0/2192710367 wait complete. 2024-03-27T08:43:01.063 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota b94fed44-a57c-48b9-b4e4-7a1f5935fb7f max_objects 10 2024-03-27T08:43:01.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.158+0000 7f582dd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:01.159 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.166+0000 7f582dd5d640 1 Processor -- start 2024-03-27T08:43:01.160 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.166+0000 7f582dd5d640 1 -- start start 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.167+0000 7f582dd5d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f582812e6a0 0x7f582812eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.167+0000 7f582dd5d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f582812e160 con 0x7f5828131660 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.167+0000 7f582dd5d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f582806c750 con 0x7f582812e6a0 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58277fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f582812e6a0 0x7f582812eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f5826ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f5828131660 0x7f5828131a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:33136/0 (socket says 172.21.15.104:33136) 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f5826ffd640 1 -- 172.21.15.104:0/185109958 learned_addr learned my addr 172.21.15.104:0/185109958 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58277fe640 1 --2- 172.21.15.104:0/185109958 >> v2:172.21.15.104:3300/0 conn(0x7f582812e6a0 0x7f582812eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51902/0 (socket says 172.21.15.104:51902) 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58277fe640 1 -- 172.21.15.104:0/185109958 >> v1:172.21.15.104:6789/0 conn(0x7f5828131660 legacy=0x7f5828131a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58277fe640 1 -- 172.21.15.104:0/185109958 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582806ae80 con 0x7f582812e6a0 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58267fc640 1 -- 172.21.15.104:0/185109958 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706356049 0 0) 0x7f582812e160 con 0x7f5828131660 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58277fe640 1 --2- 172.21.15.104:0/185109958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812eab0 crc :-1 s=READY pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a608ad406f548e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58267fc640 1 -- 172.21.15.104:0/185109958 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5818009d90 con 0x7f582812e6a0 2024-03-27T08:43:01.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58267fc640 1 -- 172.21.15.104:0/185109958 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f581800e230 con 0x7f582812e6a0 2024-03-27T08:43:01.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.168+0000 7f58267fc640 1 -- 172.21.15.104:0/185109958 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f581800e4d0 con 0x7f582812e6a0 2024-03-27T08:43:01.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 -- 172.21.15.104:0/185109958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 msgr2=0x7f582812eab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 --2- 172.21.15.104:0/185109958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812eab0 crc :-1 s=READY pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 -- 172.21.15.104:0/185109958 shutdown_connections 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 --2- 172.21.15.104:0/185109958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812eab0 unknown :-1 s=CLOSED pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 -- 172.21.15.104:0/185109958 >> 172.21.15.104:0/185109958 conn(0x7f582809fad0 msgr2=0x7f582809fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 -- 172.21.15.104:0/185109958 shutdown_connections 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 1 -- 172.21.15.104:0/185109958 wait complete. 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.169+0000 7f582dd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.171+0000 7f582dd5d640 1 Processor -- start 2024-03-27T08:43:01.164 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.171+0000 7f582dd5d640 1 -- start start 2024-03-27T08:43:01.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f582dd5d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:01.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f582dd5d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f582809ae70 con 0x7f582812e6a0 2024-03-27T08:43:01.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f58277fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:01.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f58277fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51910/0 (socket says 172.21.15.104:51910) 2024-03-27T08:43:01.166 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f58277fe640 1 -- 172.21.15.104:0/1456149528 learned_addr learned my addr 172.21.15.104:0/1456149528 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:01.166 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f58277fe640 1 -- 172.21.15.104:0/1456149528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582812d9e0 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f58277fe640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812a610 crc :-1 s=READY pgs=12923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.172+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5818013610 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.173+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582812c570 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.173+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5818013770 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.173+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5818013a10 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.173+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582812d380 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.173+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f5818013c10 con 0x7f582812e6a0 2024-03-27T08:43:01.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.173+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5828130c30 con 0x7f582812e6a0 2024-03-27T08:43:01.168 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.174+0000 7f580ffff640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5814038750 0x7f581403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:01.168 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.174+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(701..701 src has 397..701) ==== 4179+0+0 (crc 0 0 0) 0x7f5818018040 con 0x7f582812e6a0 2024-03-27T08:43:01.168 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.174+0000 7f5826ffd640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5814038750 0x7f581403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:01.168 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.174+0000 7f5826ffd640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5814038750 0x7f581403ac60 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:01.173 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.179+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f5818018c30 con 0x7f582812e6a0 2024-03-27T08:43:01.378 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:01.384+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "field": "max_objects", "val": "10"} v 0) -- 0x7f58280693c0 con 0x7f582812e6a0 2024-03-27T08:43:02.008 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:02.016+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b94fed44-a57c-48b9-b4e4-7a1f5935fb7f v702) ==== 223+0+0 (crc 0 0 0) 0x7f581800f2a0 con 0x7f582812e6a0 2024-03-27T08:43:02.115 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:02.123+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "field": "max_objects", "val": "10"} v 0) -- 0x7f58280008d0 con 0x7f582812e6a0 2024-03-27T08:43:03.054 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.060+0000 7f580ffff640 1 -- 172.21.15.104:0/1456149528 <== mon.0 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b94fed44-a57c-48b9-b4e4-7a1f5935fb7f v703) ==== 223+0+0 (crc 0 0 0) 0x7f5818018a50 con 0x7f582812e6a0 2024-03-27T08:43:03.054 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_objects = 10 for pool b94fed44-a57c-48b9-b4e4-7a1f5935fb7f 2024-03-27T08:43:03.056 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.062+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5814038750 msgr2=0x7f581403ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:03.056 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.062+0000 7f582dd5d640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5814038750 0x7f581403ac60 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:03.056 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.062+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 msgr2=0x7f582812a610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:03.056 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.062+0000 7f582dd5d640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812a610 crc :-1 s=READY pgs=12923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:03.057 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.063+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 shutdown_connections 2024-03-27T08:43:03.057 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.063+0000 7f582dd5d640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5814038750 0x7f581403ac60 unknown :-1 s=CLOSED pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:03.057 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.063+0000 7f582dd5d640 1 --2- 172.21.15.104:0/1456149528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f582812e6a0 0x7f582812a610 unknown :-1 s=CLOSED pgs=12923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:03.057 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.063+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 >> 172.21.15.104:0/1456149528 conn(0x7f582809fad0 msgr2=0x7f58281b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:03.057 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.063+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 shutdown_connections 2024-03-27T08:43:03.057 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:03.063+0000 7f582dd5d640 1 -- 172.21.15.104:0/1456149528 wait complete. 2024-03-27T08:43:03.067 INFO:tasks.workunit.client.0.smithi104.stderr:+ sleep 30 2024-03-27T08:43:33.069 INFO:tasks.workunit.client.0.smithi104.stderr:++ seq 1 10 2024-03-27T08:43:33.070 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.070 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj1 /etc/passwd 2024-03-27T08:43:33.114 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.120+0000 7fd232ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.119 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.126+0000 7fd232ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.119 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.126+0000 7fd232ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.120 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.126+0000 7fd232ec9ec0 1 Processor -- start 2024-03-27T08:43:33.120 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.126+0000 7fd232ec9ec0 1 -- start start 2024-03-27T08:43:33.123 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.127+0000 7fd232ec9ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1547ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.123 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.127+0000 7fd232ec9ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x141b0b0 con 0x14b1c90 2024-03-27T08:43:33.123 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.127+0000 7fd22b39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1547ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.123 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd22b39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1547ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34652/0 (socket says 172.21.15.104:34652) 2024-03-27T08:43:33.123 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd22b39b640 1 -- 172.21.15.104:0/1774401982 learned_addr learned my addr 172.21.15.104:0/1774401982 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.123 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd22b39b640 1 -- 172.21.15.104:0/1774401982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x13bb1e0 con 0x14b1c90 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd22b39b640 1 --2- 172.21.15.104:0/1774401982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1547ee0 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f61abc26d4986f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd229b98640 1 -- 172.21.15.104:0/1774401982 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd21400c490 con 0x14b1c90 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd229b98640 1 -- 172.21.15.104:0/1774401982 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd21400c5f0 con 0x14b1c90 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.128+0000 7fd229b98640 1 -- 172.21.15.104:0/1774401982 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd21400c890 con 0x14b1c90 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/1774401982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 msgr2=0x1547ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/1774401982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1547ee0 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/1774401982 shutdown_connections 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/1774401982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1547ee0 unknown :-1 s=CLOSED pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.124 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/1774401982 >> 172.21.15.104:0/1774401982 conn(0x13cfea0 msgr2=0x154c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/1774401982 shutdown_connections 2024-03-27T08:43:33.125 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.129+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/1774401982 wait complete. 2024-03-27T08:43:33.127 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.130+0000 7fd232ec9ec0 1 Processor -- start 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.130+0000 7fd232ec9ec0 1 -- start start 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd232ec9ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1537bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd232ec9ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1418b30 con 0x14b1c90 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd22b39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1537bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd22b39b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1537bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34668/0 (socket says 172.21.15.104:34668) 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd22b39b640 1 -- 172.21.15.104:0/98353859 learned_addr learned my addr 172.21.15.104:0/98353859 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd22b39b640 1 -- 172.21.15.104:0/98353859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x13ade30 con 0x14b1c90 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd22b39b640 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1537bb0 crc :-1 s=READY pgs=12927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd228b96640 1 -- 172.21.15.104:0/98353859 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd214016040 con 0x14b1c90 2024-03-27T08:43:33.128 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd228b96640 1 -- 172.21.15.104:0/98353859 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd214011b30 con 0x14b1c90 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x13ad890 con 0x14b1c90 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.131+0000 7fd228b96640 1 -- 172.21.15.104:0/98353859 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd214011dd0 con 0x14b1c90 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.132+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x153b390 con 0x14b1c90 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.133+0000 7fd228b96640 1 -- 172.21.15.104:0/98353859 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fd21400e440 con 0x14b1c90 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.133+0000 7fd228b96640 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd210037030 0x7fd210039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.133+0000 7fd228b96640 1 -- 172.21.15.104:0/98353859 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7fd214048e60 con 0x14b1c90 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.133+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> v2:172.21.15.177:6807/929709834 conn(0x14b3880 0x1538600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.133+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 201.7 201:ff7b1f36:::obj1:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x1539e00 con 0x14b3880 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.133+0000 7fd22ab9a640 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd210037030 0x7fd210039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.129 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.134+0000 7fd22ab9a640 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd210037030 0x7fd210039540 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.130 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.134+0000 7fd22bb9c640 1 --2- 172.21.15.104:0/98353859 >> v2:172.21.15.177:6807/929709834 conn(0x14b3880 0x1538600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.133 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.139+0000 7fd22bb9c640 1 --2- 172.21.15.104:0/98353859 >> v2:172.21.15.177:6807/929709834 conn(0x14b3880 0x1538600 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.152+0000 7fd22bb9c640 1 -- 172.21.15.104:0/98353859 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj1 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fd21c00a430 con 0x14b3880 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.152+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 >> v2:172.21.15.177:6807/929709834 conn(0x14b3880 msgr2=0x1538600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.152+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> v2:172.21.15.177:6807/929709834 conn(0x14b3880 0x1538600 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd210037030 msgr2=0x7fd210039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd210037030 0x7fd210039540 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 msgr2=0x1537bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1537bb0 crc :-1 s=READY pgs=12927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 shutdown_connections 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> v2:172.21.15.177:6807/929709834 conn(0x14b3880 0x1538600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd210037030 0x7fd210039540 unknown :-1 s=CLOSED pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 --2- 172.21.15.104:0/98353859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x14b1c90 0x1537bb0 unknown :-1 s=CLOSED pgs=12927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 >> 172.21.15.104:0/98353859 conn(0x13cfea0 msgr2=0x154c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 shutdown_connections 2024-03-27T08:43:33.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.153+0000 7fd232ec9ec0 1 -- 172.21.15.104:0/98353859 wait complete. 2024-03-27T08:43:33.149 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.149 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj2 /etc/passwd 2024-03-27T08:43:33.186 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.193+0000 7f0051854ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.192 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.198+0000 7f0051854ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.192 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.198+0000 7f0051854ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.192 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.199+0000 7f0051854ec0 1 Processor -- start 2024-03-27T08:43:33.193 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.199+0000 7f0051854ec0 1 -- start start 2024-03-27T08:43:33.194 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f0051854ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x15e8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f0051854ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x15520b0 con 0x15ecf70 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f004999b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x15e8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f004999b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x15e8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34690/0 (socket says 172.21.15.104:34690) 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f004999b640 1 -- 172.21.15.104:0/3542673056 learned_addr learned my addr 172.21.15.104:0/3542673056 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f004999b640 1 -- 172.21.15.104:0/3542673056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x14f21e0 con 0x15ecf70 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f004999b640 1 --2- 172.21.15.104:0/3542673056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x15e8a60 crc :-1 s=READY pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57283456c83f30 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f00437fe640 1 -- 172.21.15.104:0/3542673056 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f003402c390 con 0x15ecf70 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.200+0000 7f00437fe640 1 -- 172.21.15.104:0/3542673056 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f003402c4f0 con 0x15ecf70 2024-03-27T08:43:33.195 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f00437fe640 1 -- 172.21.15.104:0/3542673056 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f003402ccb0 con 0x15ecf70 2024-03-27T08:43:33.196 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 -- 172.21.15.104:0/3542673056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 msgr2=0x15e8a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3542673056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x15e8a60 crc :-1 s=READY pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 -- 172.21.15.104:0/3542673056 shutdown_connections 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3542673056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x15e8a60 unknown :-1 s=CLOSED pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 -- 172.21.15.104:0/3542673056 >> 172.21.15.104:0/3542673056 conn(0x1506ea0 msgr2=0x1705100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 -- 172.21.15.104:0/3542673056 shutdown_connections 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.201+0000 7f0051854ec0 1 -- 172.21.15.104:0/3542673056 wait complete. 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.202+0000 7f0051854ec0 1 Processor -- start 2024-03-27T08:43:33.197 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.202+0000 7f0051854ec0 1 -- start start 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.203+0000 7f0051854ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x1714cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.203+0000 7f0051854ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x154fb30 con 0x15ecf70 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.203+0000 7f004999b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x1714cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.203+0000 7f004999b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x1714cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34702/0 (socket says 172.21.15.104:34702) 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.203+0000 7f004999b640 1 -- 172.21.15.104:0/3910262438 learned_addr learned my addr 172.21.15.104:0/3910262438 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f004999b640 1 -- 172.21.15.104:0/3910262438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x14e4e30 con 0x15ecf70 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f004999b640 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x1714cf0 crc :-1 s=READY pgs=12931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f00427fc640 1 -- 172.21.15.104:0/3910262438 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0034038790 con 0x15ecf70 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f00427fc640 1 -- 172.21.15.104:0/3910262438 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f00340388f0 con 0x15ecf70 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x14e4890 con 0x15ecf70 2024-03-27T08:43:33.200 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f00427fc640 1 -- 172.21.15.104:0/3910262438 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0034038b90 con 0x15ecf70 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.204+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x16f0ea0 con 0x15ecf70 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.205+0000 7f00427fc640 1 -- 172.21.15.104:0/3910262438 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f0034038d40 con 0x15ecf70 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.205+0000 7f00427fc640 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0030037030 0x7f0030039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.205+0000 7f00427fc640 1 -- 172.21.15.104:0/3910262438 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f0034031040 con 0x15ecf70 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.205+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> v2:172.21.15.177:6805/2552358937 conn(0x15ed8c0 0x17020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.205+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 --> v2:172.21.15.177:6805/2552358937 -- osd_op(unknown.0.0:1 201.8 201:104778fc:::obj2:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x16f2500 con 0x15ed8c0 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.205+0000 7f004919a640 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0030037030 0x7f0030039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.201 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.206+0000 7f004919a640 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0030037030 0x7f0030039540 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.202 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.206+0000 7f004a19c640 1 --2- 172.21.15.104:0/3910262438 >> v2:172.21.15.177:6805/2552358937 conn(0x15ed8c0 0x17020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.204 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.211+0000 7f004a19c640 1 --2- 172.21.15.104:0/3910262438 >> v2:172.21.15.177:6805/2552358937 conn(0x15ed8c0 0x17020a0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f004a19c640 1 -- 172.21.15.104:0/3910262438 <== osd.2 v2:172.21.15.177:6805/2552358937 1 ==== osd_op_reply(1 obj2 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f003c00a430 con 0x15ed8c0 2024-03-27T08:43:33.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 >> v2:172.21.15.177:6805/2552358937 conn(0x15ed8c0 msgr2=0x17020a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> v2:172.21.15.177:6805/2552358937 conn(0x15ed8c0 0x17020a0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0030037030 msgr2=0x7f0030039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.218 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0030037030 0x7f0030039540 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 msgr2=0x1714cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x1714cf0 crc :-1 s=READY pgs=12931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 shutdown_connections 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> v2:172.21.15.177:6805/2552358937 conn(0x15ed8c0 0x17020a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0030037030 0x7f0030039540 unknown :-1 s=CLOSED pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 --2- 172.21.15.104:0/3910262438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x15ecf70 0x1714cf0 unknown :-1 s=CLOSED pgs=12931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 >> 172.21.15.104:0/3910262438 conn(0x1506ea0 msgr2=0x1705100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 shutdown_connections 2024-03-27T08:43:33.219 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.224+0000 7f0051854ec0 1 -- 172.21.15.104:0/3910262438 wait complete. 2024-03-27T08:43:33.220 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.220 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj3 /etc/passwd 2024-03-27T08:43:33.256 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.262+0000 7f186f643ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.261 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.267+0000 7f186f643ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.261 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.267+0000 7f186f643ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.261 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.268+0000 7f186f643ec0 1 Processor -- start 2024-03-27T08:43:33.262 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.268+0000 7f186f643ec0 1 -- start start 2024-03-27T08:43:33.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f186f643ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f186f643ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1fd10b0 con 0x20518f0 2024-03-27T08:43:33.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1866f9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1866f9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20661a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34716/0 (socket says 172.21.15.104:34716) 2024-03-27T08:43:33.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1866f9a640 1 -- 172.21.15.104:0/2982330943 learned_addr learned my addr 172.21.15.104:0/2982330943 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1866f9a640 1 -- 172.21.15.104:0/2982330943 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1f711e0 con 0x20518f0 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1866f9a640 1 --2- 172.21.15.104:0/2982330943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20661a0 crc :-1 s=READY pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21383155db7da896 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1865797640 1 -- 172.21.15.104:0/2982330943 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f185c029c00 con 0x20518f0 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1865797640 1 -- 172.21.15.104:0/2982330943 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f185c00a040 con 0x20518f0 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.269+0000 7f1865797640 1 -- 172.21.15.104:0/2982330943 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f185c00a2e0 con 0x20518f0 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 -- 172.21.15.104:0/2982330943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 msgr2=0x20661a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 --2- 172.21.15.104:0/2982330943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20661a0 crc :-1 s=READY pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 -- 172.21.15.104:0/2982330943 shutdown_connections 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 --2- 172.21.15.104:0/2982330943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20661a0 unknown :-1 s=CLOSED pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 -- 172.21.15.104:0/2982330943 >> 172.21.15.104:0/2982330943 conn(0x1f85ea0 msgr2=0x20ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 -- 172.21.15.104:0/2982330943 shutdown_connections 2024-03-27T08:43:33.265 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.270+0000 7f186f643ec0 1 -- 172.21.15.104:0/2982330943 wait complete. 2024-03-27T08:43:33.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.271+0000 7f186f643ec0 1 Processor -- start 2024-03-27T08:43:33.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.271+0000 7f186f643ec0 1 -- start start 2024-03-27T08:43:33.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f186f643ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f186f643ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1fceb30 con 0x20518f0 2024-03-27T08:43:33.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f1866f9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f1866f9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20fa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34726/0 (socket says 172.21.15.104:34726) 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f1866f9a640 1 -- 172.21.15.104:0/1693280160 learned_addr learned my addr 172.21.15.104:0/1693280160 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f1866f9a640 1 -- 172.21.15.104:0/1693280160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1f63e30 con 0x20518f0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f1866f9a640 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20fa8e0 crc :-1 s=READY pgs=12935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f184ffff640 1 -- 172.21.15.104:0/1693280160 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f185c03b040 con 0x20518f0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.272+0000 7f184ffff640 1 -- 172.21.15.104:0/1693280160 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f185c006c60 con 0x20518f0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.273+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1f63890 con 0x20518f0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.273+0000 7f184ffff640 1 -- 172.21.15.104:0/1693280160 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f185c029c00 con 0x20518f0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.273+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2102c20 con 0x20518f0 2024-03-27T08:43:33.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.273+0000 7f184ffff640 1 -- 172.21.15.104:0/1693280160 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f185c035450 con 0x20518f0 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.274+0000 7f184ffff640 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1848037030 0x7f1848039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.274+0000 7f184ffff640 1 -- 172.21.15.104:0/1693280160 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f185c02e040 con 0x20518f0 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.274+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> v2:172.21.15.104:6801/1224298329 conn(0x2052240 0x2196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.274+0000 7f1866799640 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1848037030 0x7f1848039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.274+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 201.1 201:8dd16f86:::obj3:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x20e83d0 con 0x2052240 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.275+0000 7f186779b640 1 --2- 172.21.15.104:0/1693280160 >> v2:172.21.15.104:6801/1224298329 conn(0x2052240 0x2196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.270 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.275+0000 7f1866799640 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1848037030 0x7f1848039540 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.272 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.278+0000 7f186779b640 1 --2- 172.21.15.104:0/1693280160 >> v2:172.21.15.104:6801/1224298329 conn(0x2052240 0x2196340 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.284 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186779b640 1 -- 172.21.15.104:0/1693280160 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj3 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f185800a430 con 0x2052240 2024-03-27T08:43:33.284 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 >> v2:172.21.15.104:6801/1224298329 conn(0x2052240 msgr2=0x2196340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.284 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> v2:172.21.15.104:6801/1224298329 conn(0x2052240 0x2196340 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.284 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1848037030 msgr2=0x7f1848039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1848037030 0x7f1848039540 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 msgr2=0x20fa8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20fa8e0 crc :-1 s=READY pgs=12935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 shutdown_connections 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1848037030 0x7f1848039540 unknown :-1 s=CLOSED pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> v2:172.21.15.104:6801/1224298329 conn(0x2052240 0x2196340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 --2- 172.21.15.104:0/1693280160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x20518f0 0x20fa8e0 unknown :-1 s=CLOSED pgs=12935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 >> 172.21.15.104:0/1693280160 conn(0x1f85ea0 msgr2=0x20fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 shutdown_connections 2024-03-27T08:43:33.285 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.290+0000 7f186f643ec0 1 -- 172.21.15.104:0/1693280160 wait complete. 2024-03-27T08:43:33.286 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.286 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj4 /etc/passwd 2024-03-27T08:43:33.321 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.328+0000 7f83d9c8aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.332+0000 7f83d9c8aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.326 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.333+0000 7f83d9c8aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.333+0000 7f83d9c8aec0 1 Processor -- start 2024-03-27T08:43:33.327 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.333+0000 7f83d9c8aec0 1 -- start start 2024-03-27T08:43:33.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.334+0000 7f83d9c8aec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1ff9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.334+0000 7f83d9c8aec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1eca0b0 con 0x1f64b20 2024-03-27T08:43:33.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.334+0000 7f83d1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1ff9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.334+0000 7f83d1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1ff9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34746/0 (socket says 172.21.15.104:34746) 2024-03-27T08:43:33.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.334+0000 7f83d1d9b640 1 -- 172.21.15.104:0/2070053644 learned_addr learned my addr 172.21.15.104:0/2070053644 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d1d9b640 1 -- 172.21.15.104:0/2070053644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1e6a1e0 con 0x1f64b20 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d1d9b640 1 --2- 172.21.15.104:0/2070053644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1ff9b70 crc :-1 s=READY pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9459b18aca72b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83cb7fe640 1 -- 172.21.15.104:0/2070053644 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f83bc0061e0 con 0x1f64b20 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83cb7fe640 1 -- 172.21.15.104:0/2070053644 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f83bc004690 con 0x1f64b20 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83cb7fe640 1 -- 172.21.15.104:0/2070053644 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f83bc006340 con 0x1f64b20 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/2070053644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 msgr2=0x1ff9b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/2070053644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1ff9b70 crc :-1 s=READY pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/2070053644 shutdown_connections 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/2070053644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1ff9b70 unknown :-1 s=CLOSED pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.331 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.335+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/2070053644 >> 172.21.15.104:0/2070053644 conn(0x1e7eea0 msgr2=0x1f5e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.332 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.336+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/2070053644 shutdown_connections 2024-03-27T08:43:33.332 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.336+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/2070053644 wait complete. 2024-03-27T08:43:33.332 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.336+0000 7f83d9c8aec0 1 Processor -- start 2024-03-27T08:43:33.332 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.336+0000 7f83d9c8aec0 1 -- start start 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.337+0000 7f83d9c8aec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1fe2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.337+0000 7f83d9c8aec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1ec7b30 con 0x1f64b20 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.337+0000 7f83d1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1fe2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.337+0000 7f83d1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1fe2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34750/0 (socket says 172.21.15.104:34750) 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.337+0000 7f83d1d9b640 1 -- 172.21.15.104:0/3667660936 learned_addr learned my addr 172.21.15.104:0/3667660936 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83d1d9b640 1 -- 172.21.15.104:0/3667660936 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1e5ce30 con 0x1f64b20 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83d1d9b640 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1fe2160 crc :-1 s=READY pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83ca7fc640 1 -- 172.21.15.104:0/3667660936 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f83bc0019d0 con 0x1f64b20 2024-03-27T08:43:33.335 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83ca7fc640 1 -- 172.21.15.104:0/3667660936 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f83bc001b30 con 0x1f64b20 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1e5c890 con 0x1f64b20 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83ca7fc640 1 -- 172.21.15.104:0/3667660936 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f83bc001dd0 con 0x1f64b20 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.338+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1fe5640 con 0x1f64b20 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.339+0000 7f83ca7fc640 1 -- 172.21.15.104:0/3667660936 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f83bc001f80 con 0x1f64b20 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.339+0000 7f83ca7fc640 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f83ac037030 0x7f83ac039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.339+0000 7f83ca7fc640 1 -- 172.21.15.104:0/3667660936 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f83bc048210 con 0x1f64b20 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.339+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> v2:172.21.15.177:6807/929709834 conn(0x1f65640 0x1ff56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.339+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 201.0 201:0ee9ae15:::obj4:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x1fe5c50 con 0x1f65640 2024-03-27T08:43:33.336 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.339+0000 7f83d159a640 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f83ac037030 0x7f83ac039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.337 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.340+0000 7f83d159a640 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f83ac037030 0x7f83ac039540 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.337 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.341+0000 7f83d259c640 1 --2- 172.21.15.104:0/3667660936 >> v2:172.21.15.177:6807/929709834 conn(0x1f65640 0x1ff56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.339 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.345+0000 7f83d259c640 1 --2- 172.21.15.104:0/3667660936 >> v2:172.21.15.177:6807/929709834 conn(0x1f65640 0x1ff56e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d259c640 1 -- 172.21.15.104:0/3667660936 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj4 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f83c400c430 con 0x1f65640 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 >> v2:172.21.15.177:6807/929709834 conn(0x1f65640 msgr2=0x1ff56e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> v2:172.21.15.177:6807/929709834 conn(0x1f65640 0x1ff56e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f83ac037030 msgr2=0x7f83ac039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f83ac037030 0x7f83ac039540 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 msgr2=0x1fe2160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1fe2160 crc :-1 s=READY pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 shutdown_connections 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> v2:172.21.15.177:6807/929709834 conn(0x1f65640 0x1ff56e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f83ac037030 0x7f83ac039540 unknown :-1 s=CLOSED pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 --2- 172.21.15.104:0/3667660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1f64b20 0x1fe2160 unknown :-1 s=CLOSED pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 >> 172.21.15.104:0/3667660936 conn(0x1e7eea0 msgr2=0x1f5e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.353 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 shutdown_connections 2024-03-27T08:43:33.354 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.358+0000 7f83d9c8aec0 1 -- 172.21.15.104:0/3667660936 wait complete. 2024-03-27T08:43:33.354 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.354 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj5 /etc/passwd 2024-03-27T08:43:33.389 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.395+0000 7f1f61a32ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.394 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.400+0000 7f1f61a32ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.394 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.400+0000 7f1f61a32ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.395 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.401+0000 7f1f61a32ec0 1 Processor -- start 2024-03-27T08:43:33.395 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.401+0000 7f1f61a32ec0 1 -- start start 2024-03-27T08:43:33.396 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f61a32ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc97d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.396 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f61a32ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xb680b0 con 0xc02f70 2024-03-27T08:43:33.396 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f59b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc97d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.396 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f59b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc97d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34768/0 (socket says 172.21.15.104:34768) 2024-03-27T08:43:33.397 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f59b9b640 1 -- 172.21.15.104:0/1211218031 learned_addr learned my addr 172.21.15.104:0/1211218031 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.397 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f59b9b640 1 -- 172.21.15.104:0/1211218031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb081e0 con 0xc02f70 2024-03-27T08:43:33.397 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f59b9b640 1 --2- 172.21.15.104:0/1211218031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc97d40 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc88e1f08b322f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.397 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f537fe640 1 -- 172.21.15.104:0/1211218031 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1f44029ad0 con 0xc02f70 2024-03-27T08:43:33.397 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f537fe640 1 -- 172.21.15.104:0/1211218031 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1f4400a040 con 0xc02f70 2024-03-27T08:43:33.397 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.402+0000 7f1f537fe640 1 -- 172.21.15.104:0/1211218031 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1f4400a1b0 con 0xc02f70 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/1211218031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 msgr2=0xc97d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/1211218031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc97d40 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/1211218031 shutdown_connections 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/1211218031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc97d40 unknown :-1 s=CLOSED pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/1211218031 >> 172.21.15.104:0/1211218031 conn(0xb1cea0 msgr2=0xc94f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/1211218031 shutdown_connections 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.403+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/1211218031 wait complete. 2024-03-27T08:43:33.398 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.404+0000 7f1f61a32ec0 1 Processor -- start 2024-03-27T08:43:33.399 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.404+0000 7f1f61a32ec0 1 -- start start 2024-03-27T08:43:33.400 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f61a32ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc7f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f61a32ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xb65b30 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f59b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc7f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f59b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc7f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34772/0 (socket says 172.21.15.104:34772) 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f59b9b640 1 -- 172.21.15.104:0/3783309667 learned_addr learned my addr 172.21.15.104:0/3783309667 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f59b9b640 1 -- 172.21.15.104:0/3783309667 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xafae30 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f59b9b640 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc7f510 crc :-1 s=READY pgs=12943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f527fc640 1 -- 172.21.15.104:0/3783309667 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1f4403b040 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.405+0000 7f1f527fc640 1 -- 172.21.15.104:0/3783309667 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1f4400a1b0 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.406+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xafa890 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.406+0000 7f1f527fc640 1 -- 172.21.15.104:0/3783309667 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1f44006c70 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.406+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0xbff620 con 0xc02f70 2024-03-27T08:43:33.401 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.406+0000 7f1f527fc640 1 -- 172.21.15.104:0/3783309667 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f1f44035450 con 0xc02f70 2024-03-27T08:43:33.402 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.407+0000 7f1f527fc640 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1f40037030 0x7f1f40039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.402 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.407+0000 7f1f527fc640 1 -- 172.21.15.104:0/3783309667 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f1f4402e040 con 0xc02f70 2024-03-27T08:43:33.402 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.407+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> v2:172.21.15.104:6804/1714017423 conn(0xc038c0 0xd2f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.402 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.407+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 --> v2:172.21.15.104:6804/1714017423 -- osd_op(unknown.0.0:1 201.3 201:c52c9666:::obj5:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0xc85e70 con 0xc038c0 2024-03-27T08:43:33.402 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.407+0000 7f1f5939a640 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1f40037030 0x7f1f40039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.402 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.408+0000 7f1f5939a640 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1f40037030 0x7f1f40039540 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.403 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.408+0000 7f1f5a39c640 1 --2- 172.21.15.104:0/3783309667 >> v2:172.21.15.104:6804/1714017423 conn(0xc038c0 0xd2f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.405 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.411+0000 7f1f5a39c640 1 --2- 172.21.15.104:0/3783309667 >> v2:172.21.15.104:6804/1714017423 conn(0xc038c0 0xd2f450 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.417 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f5a39c640 1 -- 172.21.15.104:0/3783309667 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== osd_op_reply(1 obj5 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f1f4c00a430 con 0xc038c0 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 >> v2:172.21.15.104:6804/1714017423 conn(0xc038c0 msgr2=0xd2f450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> v2:172.21.15.104:6804/1714017423 conn(0xc038c0 0xd2f450 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1f40037030 msgr2=0x7f1f40039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1f40037030 0x7f1f40039540 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 msgr2=0xc7f510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc7f510 crc :-1 s=READY pgs=12943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 shutdown_connections 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> v2:172.21.15.104:6804/1714017423 conn(0xc038c0 0xd2f450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f1f40037030 0x7f1f40039540 unknown :-1 s=CLOSED pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 --2- 172.21.15.104:0/3783309667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc02f70 0xc7f510 unknown :-1 s=CLOSED pgs=12943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 >> 172.21.15.104:0/3783309667 conn(0xb1cea0 msgr2=0xc94f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 shutdown_connections 2024-03-27T08:43:33.418 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.423+0000 7f1f61a32ec0 1 -- 172.21.15.104:0/3783309667 wait complete. 2024-03-27T08:43:33.419 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.419 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj6 /etc/passwd 2024-03-27T08:43:33.454 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.460+0000 7fe1f76fcec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.459 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.465+0000 7fe1f76fcec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.459 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.465+0000 7fe1f76fcec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.460 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.466+0000 7fe1f76fcec0 1 Processor -- start 2024-03-27T08:43:33.460 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.466+0000 7fe1f76fcec0 1 -- start start 2024-03-27T08:43:33.461 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.467+0000 7fe1f76fcec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.467+0000 7fe1f76fcec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x13ca0b0 con 0x1460c90 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.467+0000 7fe1eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.467+0000 7fe1eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34802/0 (socket says 172.21.15.104:34802) 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.467+0000 7fe1eaffd640 1 -- 172.21.15.104:0/3216923470 learned_addr learned my addr 172.21.15.104:0/3216923470 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.467+0000 7fe1eaffd640 1 -- 172.21.15.104:0/3216923470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x136a1e0 con 0x1460c90 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1eaffd640 1 --2- 172.21.15.104:0/3216923470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f36c0 crc :-1 s=READY pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a923d43fb9fa938a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1e97fa640 1 -- 172.21.15.104:0/3216923470 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1e400b5d0 con 0x1460c90 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1e97fa640 1 -- 172.21.15.104:0/3216923470 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe1e400b730 con 0x1460c90 2024-03-27T08:43:33.462 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1e97fa640 1 -- 172.21.15.104:0/3216923470 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1e400b9d0 con 0x1460c90 2024-03-27T08:43:33.463 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/3216923470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 msgr2=0x14f36c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.463 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/3216923470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f36c0 crc :-1 s=READY pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.463 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/3216923470 shutdown_connections 2024-03-27T08:43:33.464 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/3216923470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f36c0 unknown :-1 s=CLOSED pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.464 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/3216923470 >> 172.21.15.104:0/3216923470 conn(0x137eea0 msgr2=0x14f6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.464 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.468+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/3216923470 shutdown_connections 2024-03-27T08:43:33.464 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.469+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/3216923470 wait complete. 2024-03-27T08:43:33.464 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.469+0000 7fe1f76fcec0 1 Processor -- start 2024-03-27T08:43:33.464 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.469+0000 7fe1f76fcec0 1 -- start start 2024-03-27T08:43:33.465 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.470+0000 7fe1f76fcec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.465 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.470+0000 7fe1f76fcec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x13c7b30 con 0x1460c90 2024-03-27T08:43:33.465 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.470+0000 7fe1eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.465 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.470+0000 7fe1eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34818/0 (socket says 172.21.15.104:34818) 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.470+0000 7fe1eaffd640 1 -- 172.21.15.104:0/1057597856 learned_addr learned my addr 172.21.15.104:0/1057597856 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1eaffd640 1 -- 172.21.15.104:0/1057597856 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x135ce30 con 0x1460c90 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1eaffd640 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f3b10 crc :-1 s=READY pgs=12947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1d3fff640 1 -- 172.21.15.104:0/1057597856 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1e40129c0 con 0x1460c90 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1d3fff640 1 -- 172.21.15.104:0/1057597856 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe1e4005040 con 0x1460c90 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x135c890 con 0x1460c90 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1d3fff640 1 -- 172.21.15.104:0/1057597856 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1e4012c50 con 0x1460c90 2024-03-27T08:43:33.466 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.471+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1586090 con 0x1460c90 2024-03-27T08:43:33.467 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.472+0000 7fe1d3fff640 1 -- 172.21.15.104:0/1057597856 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe1e400d040 con 0x1460c90 2024-03-27T08:43:33.467 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.472+0000 7fe1d3fff640 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe1d8037030 0x7fe1d8039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.467 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.472+0000 7fe1d3fff640 1 -- 172.21.15.104:0/1057597856 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7fe1e400a040 con 0x1460c90 2024-03-27T08:43:33.467 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.472+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> v2:172.21.15.104:6801/1224298329 conn(0x1462880 0x144ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.467 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.472+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 201.4 201:233a42c1:::obj6:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x145f660 con 0x1462880 2024-03-27T08:43:33.468 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.472+0000 7fe1ea7fc640 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe1d8037030 0x7fe1d8039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.468 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.473+0000 7fe1ea7fc640 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe1d8037030 0x7fe1d8039540 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.468 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.473+0000 7fe1eb7fe640 1 --2- 172.21.15.104:0/1057597856 >> v2:172.21.15.104:6801/1224298329 conn(0x1462880 0x144ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.469 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.476+0000 7fe1eb7fe640 1 --2- 172.21.15.104:0/1057597856 >> v2:172.21.15.104:6801/1224298329 conn(0x1462880 0x144ae00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1eb7fe640 1 -- 172.21.15.104:0/1057597856 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj6 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fe1dc00a430 con 0x1462880 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 >> v2:172.21.15.104:6801/1224298329 conn(0x1462880 msgr2=0x144ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> v2:172.21.15.104:6801/1224298329 conn(0x1462880 0x144ae00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe1d8037030 msgr2=0x7fe1d8039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe1d8037030 0x7fe1d8039540 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 msgr2=0x14f3b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f3b10 crc :-1 s=READY pgs=12947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.482 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 shutdown_connections 2024-03-27T08:43:33.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe1d8037030 0x7fe1d8039540 unknown :-1 s=CLOSED pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> v2:172.21.15.104:6801/1224298329 conn(0x1462880 0x144ae00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 --2- 172.21.15.104:0/1057597856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1460c90 0x14f3b10 unknown :-1 s=CLOSED pgs=12947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 >> 172.21.15.104:0/1057597856 conn(0x137eea0 msgr2=0x14fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 shutdown_connections 2024-03-27T08:43:33.483 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.487+0000 7fe1f76fcec0 1 -- 172.21.15.104:0/1057597856 wait complete. 2024-03-27T08:43:33.484 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.484 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj7 /etc/passwd 2024-03-27T08:43:33.518 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.524+0000 7fe09892aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.523 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.529+0000 7fe09892aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.523 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.529+0000 7fe09892aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.524 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.530+0000 7fe09892aec0 1 Processor -- start 2024-03-27T08:43:33.524 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.530+0000 7fe09892aec0 1 -- start start 2024-03-27T08:43:33.525 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe09892aec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xcf30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.525 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe09892aec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xbc40b0 con 0xc448f0 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe090d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xcf30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe090d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xcf30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34840/0 (socket says 172.21.15.104:34840) 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe090d9b640 1 -- 172.21.15.104:0/220740298 learned_addr learned my addr 172.21.15.104:0/220740298 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe090d9b640 1 -- 172.21.15.104:0/220740298 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb641e0 con 0xc448f0 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.531+0000 7fe090d9b640 1 --2- 172.21.15.104:0/220740298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xcf30d0 crc :-1 s=READY pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22f149ff92d9065 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe08affd640 1 -- 172.21.15.104:0/220740298 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe07c00b5d0 con 0xc448f0 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe08affd640 1 -- 172.21.15.104:0/220740298 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe07c00b730 con 0xc448f0 2024-03-27T08:43:33.526 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe08affd640 1 -- 172.21.15.104:0/220740298 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe07c00b9d0 con 0xc448f0 2024-03-27T08:43:33.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 -- 172.21.15.104:0/220740298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 msgr2=0xcf30d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 --2- 172.21.15.104:0/220740298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xcf30d0 crc :-1 s=READY pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 -- 172.21.15.104:0/220740298 shutdown_connections 2024-03-27T08:43:33.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 --2- 172.21.15.104:0/220740298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xcf30d0 unknown :-1 s=CLOSED pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.527 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 -- 172.21.15.104:0/220740298 >> 172.21.15.104:0/220740298 conn(0xb78ea0 msgr2=0xcf1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.528 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 -- 172.21.15.104:0/220740298 shutdown_connections 2024-03-27T08:43:33.528 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.532+0000 7fe09892aec0 1 -- 172.21.15.104:0/220740298 wait complete. 2024-03-27T08:43:33.528 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.533+0000 7fe09892aec0 1 Processor -- start 2024-03-27T08:43:33.528 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.533+0000 7fe09892aec0 1 -- start start 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.534+0000 7fe09892aec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xc57db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.534+0000 7fe09892aec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0xbc1b30 con 0xc448f0 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.534+0000 7fe090d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xc57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.534+0000 7fe090d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xc57db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34842/0 (socket says 172.21.15.104:34842) 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.534+0000 7fe090d9b640 1 -- 172.21.15.104:0/3083065889 learned_addr learned my addr 172.21.15.104:0/3083065889 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.534+0000 7fe090d9b640 1 -- 172.21.15.104:0/3083065889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb56e30 con 0xc448f0 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.535+0000 7fe090d9b640 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xc57db0 crc :-1 s=READY pgs=12951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.535+0000 7fe089ffb640 1 -- 172.21.15.104:0/3083065889 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe07c0129e0 con 0xc448f0 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.535+0000 7fe089ffb640 1 -- 172.21.15.104:0/3083065889 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe07c005040 con 0xc448f0 2024-03-27T08:43:33.530 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.535+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xb56890 con 0xc448f0 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.535+0000 7fe089ffb640 1 -- 172.21.15.104:0/3083065889 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe07c012c70 con 0xc448f0 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.535+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0xcf06b0 con 0xc448f0 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.536+0000 7fe089ffb640 1 -- 172.21.15.104:0/3083065889 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe07c00d040 con 0xc448f0 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.536+0000 7fe089ffb640 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe06c037030 0x7fe06c039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.536+0000 7fe089ffb640 1 -- 172.21.15.104:0/3083065889 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7fe07c00a040 con 0xc448f0 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.536+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> v2:172.21.15.104:6801/1224298329 conn(0xc45410 0xd89240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.536+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 201.9 201:9a09113a:::obj7:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0xcdac50 con 0xc45410 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.536+0000 7fe0837fe640 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe06c037030 0x7fe06c039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.531 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.537+0000 7fe0837fe640 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe06c037030 0x7fe06c039540 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.532 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.537+0000 7fe09159c640 1 --2- 172.21.15.104:0/3083065889 >> v2:172.21.15.104:6801/1224298329 conn(0xc45410 0xd89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.534 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.540+0000 7fe09159c640 1 --2- 172.21.15.104:0/3083065889 >> v2:172.21.15.104:6801/1224298329 conn(0xc45410 0xd89240 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09159c640 1 -- 172.21.15.104:0/3083065889 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj7 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fe08400a430 con 0xc45410 2024-03-27T08:43:33.546 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 >> v2:172.21.15.104:6801/1224298329 conn(0xc45410 msgr2=0xd89240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> v2:172.21.15.104:6801/1224298329 conn(0xc45410 0xd89240 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe06c037030 msgr2=0x7fe06c039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe06c037030 0x7fe06c039540 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 msgr2=0xc57db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xc57db0 crc :-1 s=READY pgs=12951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 shutdown_connections 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fe06c037030 0x7fe06c039540 unknown :-1 s=CLOSED pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> v2:172.21.15.104:6801/1224298329 conn(0xc45410 0xd89240 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 --2- 172.21.15.104:0/3083065889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0xc448f0 0xc57db0 unknown :-1 s=CLOSED pgs=12951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 >> 172.21.15.104:0/3083065889 conn(0xb78ea0 msgr2=0xcf1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 shutdown_connections 2024-03-27T08:43:33.547 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.552+0000 7fe09892aec0 1 -- 172.21.15.104:0/3083065889 wait complete. 2024-03-27T08:43:33.548 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.549 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj8 /etc/passwd 2024-03-27T08:43:33.583 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.589+0000 7f2b2284fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.594+0000 7f2b2284fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.588 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.594+0000 7f2b2284fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.595+0000 7f2b2284fec0 1 Processor -- start 2024-03-27T08:43:33.589 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.595+0000 7f2b2284fec0 1 -- start start 2024-03-27T08:43:33.590 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.596+0000 7f2b2284fec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.596+0000 7f2b2284fec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1b200b0 con 0x1ba08f0 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.596+0000 7f2b1a99b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.596+0000 7f2b1a99b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c4a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34852/0 (socket says 172.21.15.104:34852) 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.596+0000 7f2b1a99b640 1 -- 172.21.15.104:0/2093570522 learned_addr learned my addr 172.21.15.104:0/2093570522 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.596+0000 7f2b1a99b640 1 -- 172.21.15.104:0/2093570522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1ac01e0 con 0x1ba08f0 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b1a99b640 1 --2- 172.21.15.104:0/2093570522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c4a6d0 crc :-1 s=READY pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9092db95f0d39b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b19198640 1 -- 172.21.15.104:0/2093570522 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2b04029d20 con 0x1ba08f0 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b19198640 1 -- 172.21.15.104:0/2093570522 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2b040333d0 con 0x1ba08f0 2024-03-27T08:43:33.591 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b19198640 1 -- 172.21.15.104:0/2093570522 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2b04033670 con 0x1ba08f0 2024-03-27T08:43:33.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2093570522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 msgr2=0x1c4a6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.592 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2093570522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c4a6d0 crc :-1 s=READY pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2093570522 shutdown_connections 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2093570522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c4a6d0 unknown :-1 s=CLOSED pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2093570522 >> 172.21.15.104:0/2093570522 conn(0x1ad4ea0 msgr2=0x1c4b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2093570522 shutdown_connections 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.597+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2093570522 wait complete. 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.598+0000 7f2b2284fec0 1 Processor -- start 2024-03-27T08:43:33.593 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.598+0000 7f2b2284fec0 1 -- start start 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.599+0000 7f2b2284fec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c49ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.599+0000 7f2b2284fec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1b1db30 con 0x1ba08f0 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.599+0000 7f2b1a99b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c49ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.599+0000 7f2b1a99b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c49ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34856/0 (socket says 172.21.15.104:34856) 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.599+0000 7f2b1a99b640 1 -- 172.21.15.104:0/2159983451 learned_addr learned my addr 172.21.15.104:0/2159983451 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b1a99b640 1 -- 172.21.15.104:0/2159983451 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1ab2e30 con 0x1ba08f0 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b1a99b640 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c49ed0 crc :-1 s=READY pgs=12955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.595 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b137fe640 1 -- 172.21.15.104:0/2159983451 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2b04039040 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b137fe640 1 -- 172.21.15.104:0/2159983451 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2b04033df0 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1ab2890 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b137fe640 1 -- 172.21.15.104:0/2159983451 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2b04034570 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.600+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1c4b3a0 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.601+0000 7f2b137fe640 1 -- 172.21.15.104:0/2159983451 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2b0403c040 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.601+0000 7f2b137fe640 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2af4036fe0 0x7f2af40394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.601+0000 7f2b137fe640 1 -- 172.21.15.104:0/2159983451 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f2b0402e040 con 0x1ba08f0 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.601+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> v2:172.21.15.104:6801/1224298329 conn(0x1ba1240 0x1c36590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.601+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 201.5 201:a557efb1:::obj8:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x1c39430 con 0x1ba1240 2024-03-27T08:43:33.596 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.601+0000 7f2b1a19a640 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2af4036fe0 0x7f2af40394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.597 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.602+0000 7f2b1a19a640 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2af4036fe0 0x7f2af40394f0 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.597 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.602+0000 7f2b1b19c640 1 --2- 172.21.15.104:0/2159983451 >> v2:172.21.15.104:6801/1224298329 conn(0x1ba1240 0x1c36590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.599 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.605+0000 7f2b1b19c640 1 --2- 172.21.15.104:0/2159983451 >> v2:172.21.15.104:6801/1224298329 conn(0x1ba1240 0x1c36590 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.611 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b1b19c640 1 -- 172.21.15.104:0/2159983451 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj8 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f2b0c00c430 con 0x1ba1240 2024-03-27T08:43:33.611 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 >> v2:172.21.15.104:6801/1224298329 conn(0x1ba1240 msgr2=0x1c36590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> v2:172.21.15.104:6801/1224298329 conn(0x1ba1240 0x1c36590 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2af4036fe0 msgr2=0x7f2af40394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2af4036fe0 0x7f2af40394f0 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 msgr2=0x1c49ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c49ed0 crc :-1 s=READY pgs=12955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 shutdown_connections 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f2af4036fe0 0x7f2af40394f0 unknown :-1 s=CLOSED pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> v2:172.21.15.104:6801/1224298329 conn(0x1ba1240 0x1c36590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 --2- 172.21.15.104:0/2159983451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1ba08f0 0x1c49ed0 unknown :-1 s=CLOSED pgs=12955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 >> 172.21.15.104:0/2159983451 conn(0x1ad4ea0 msgr2=0x1c4b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 shutdown_connections 2024-03-27T08:43:33.612 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.617+0000 7f2b2284fec0 1 -- 172.21.15.104:0/2159983451 wait complete. 2024-03-27T08:43:33.613 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.613 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj9 /etc/passwd 2024-03-27T08:43:33.648 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.654+0000 7f6921c36ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.653 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.660+0000 7f6921c36ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.653 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.660+0000 7f6921c36ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.654 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.660+0000 7f6921c36ec0 1 Processor -- start 2024-03-27T08:43:33.654 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.660+0000 7f6921c36ec0 1 -- start start 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.661+0000 7f6921c36ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1ddd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.661+0000 7f6921c36ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1cc80b0 con 0x1d5c0a0 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.661+0000 7f691959a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1ddd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.661+0000 7f691959a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1ddd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34878/0 (socket says 172.21.15.104:34878) 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.661+0000 7f691959a640 1 -- 172.21.15.104:0/2251304402 learned_addr learned my addr 172.21.15.104:0/2251304402 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.662+0000 7f691959a640 1 -- 172.21.15.104:0/2251304402 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1c681e0 con 0x1d5c0a0 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.662+0000 7f691959a640 1 --2- 172.21.15.104:0/2251304402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1ddd760 crc :-1 s=READY pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee5775a5a693799 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.662+0000 7f690b7fe640 1 -- 172.21.15.104:0/2251304402 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f690400c490 con 0x1d5c0a0 2024-03-27T08:43:33.657 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.662+0000 7f690b7fe640 1 -- 172.21.15.104:0/2251304402 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f690400c5f0 con 0x1d5c0a0 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.662+0000 7f690b7fe640 1 -- 172.21.15.104:0/2251304402 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f690400c890 con 0x1d5c0a0 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 -- 172.21.15.104:0/2251304402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 msgr2=0x1ddd760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/2251304402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1ddd760 crc :-1 s=READY pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 -- 172.21.15.104:0/2251304402 shutdown_connections 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/2251304402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1ddd760 unknown :-1 s=CLOSED pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 -- 172.21.15.104:0/2251304402 >> 172.21.15.104:0/2251304402 conn(0x1c7cea0 msgr2=0x1df9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 -- 172.21.15.104:0/2251304402 shutdown_connections 2024-03-27T08:43:33.658 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 -- 172.21.15.104:0/2251304402 wait complete. 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 Processor -- start 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.663+0000 7f6921c36ec0 1 -- start start 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.664+0000 7f6921c36ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1de08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.664+0000 7f6921c36ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1cc5b30 con 0x1d5c0a0 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f691959a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1de08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f691959a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1de08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34882/0 (socket says 172.21.15.104:34882) 2024-03-27T08:43:33.662 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f691959a640 1 -- 172.21.15.104:0/3170230018 learned_addr learned my addr 172.21.15.104:0/3170230018 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f691959a640 1 -- 172.21.15.104:0/3170230018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1c5ae30 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f691959a640 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1de08a0 crc :-1 s=READY pgs=12959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f690a7fc640 1 -- 172.21.15.104:0/3170230018 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6904016040 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f690a7fc640 1 -- 172.21.15.104:0/3170230018 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6904011b00 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1c5a890 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f690a7fc640 1 -- 172.21.15.104:0/3170230018 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6904011da0 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.665+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1de19d0 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.666+0000 7f690a7fc640 1 -- 172.21.15.104:0/3170230018 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f690400e440 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.666+0000 7f690a7fc640 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f68f8036fe0 0x7f68f80394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.666+0000 7f690a7fc640 1 -- 172.21.15.104:0/3170230018 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f6904048fb0 con 0x1d5c0a0 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.666+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> v2:172.21.15.177:6807/929709834 conn(0x1d63230 0x1e8d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.663 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.666+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 --> v2:172.21.15.177:6807/929709834 -- osd_op(unknown.0.0:1 201.7 201:fe351e27:::obj9:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x1e8d9c0 con 0x1d63230 2024-03-27T08:43:33.664 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.667+0000 7f6918d99640 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f68f8036fe0 0x7f68f80394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.664 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.667+0000 7f6918d99640 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f68f8036fe0 0x7f68f80394f0 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.664 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.667+0000 7f6919d9b640 1 --2- 172.21.15.104:0/3170230018 >> v2:172.21.15.177:6807/929709834 conn(0x1d63230 0x1e8d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.664 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.670+0000 7f6919d9b640 1 --2- 172.21.15.104:0/3170230018 >> v2:172.21.15.177:6807/929709834 conn(0x1d63230 0x1e8d480 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6919d9b640 1 -- 172.21.15.104:0/3170230018 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== osd_op_reply(1 obj9 [writefull 0~11812] v704'2 uv2 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f690c00a430 con 0x1d63230 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 >> v2:172.21.15.177:6807/929709834 conn(0x1d63230 msgr2=0x1e8d480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> v2:172.21.15.177:6807/929709834 conn(0x1d63230 0x1e8d480 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f68f8036fe0 msgr2=0x7f68f80394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f68f8036fe0 0x7f68f80394f0 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 msgr2=0x1de08a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.677 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1de08a0 crc :-1 s=READY pgs=12959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.678 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 shutdown_connections 2024-03-27T08:43:33.678 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> v2:172.21.15.177:6807/929709834 conn(0x1d63230 0x1e8d480 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.678 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f68f8036fe0 0x7f68f80394f0 unknown :-1 s=CLOSED pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.678 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 --2- 172.21.15.104:0/3170230018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1d5c0a0 0x1de08a0 unknown :-1 s=CLOSED pgs=12959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.678 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.682+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 >> 172.21.15.104:0/3170230018 conn(0x1c7cea0 msgr2=0x1df9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.678 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.683+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 shutdown_connections 2024-03-27T08:43:33.679 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.683+0000 7f6921c36ec0 1 -- 172.21.15.104:0/3170230018 wait complete. 2024-03-27T08:43:33.679 INFO:tasks.workunit.client.0.smithi104.stderr:+ for f in `seq 1 10` 2024-03-27T08:43:33.679 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p b94fed44-a57c-48b9-b4e4-7a1f5935fb7f put obj10 /etc/passwd 2024-03-27T08:43:33.714 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.720+0000 7f5860833ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.721 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.727+0000 7f5860833ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.721 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.727+0000 7f5860833ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:43:33.721 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.727+0000 7f5860833ec0 1 Processor -- start 2024-03-27T08:43:33.721 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.728+0000 7f5860833ec0 1 -- start start 2024-03-27T08:43:33.723 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.728+0000 7f5860833ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.723 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f5860833ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x12880b0 con 0x1322f70 2024-03-27T08:43:33.723 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f585899b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f585899b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34898/0 (socket says 172.21.15.104:34898) 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f585899b640 1 -- 172.21.15.104:0/1149382479 learned_addr learned my addr 172.21.15.104:0/1149382479 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f585899b640 1 -- 172.21.15.104:0/1149382479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x12281e0 con 0x1322f70 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f585899b640 1 --2- 172.21.15.104:0/1149382479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4d20 crc :-1 s=READY pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29acc3fdf43c4f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f58527fc640 1 -- 172.21.15.104:0/1149382479 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5844029920 con 0x1322f70 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f58527fc640 1 -- 172.21.15.104:0/1149382479 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f58440333d0 con 0x1322f70 2024-03-27T08:43:33.724 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.729+0000 7f58527fc640 1 -- 172.21.15.104:0/1149382479 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5844033670 con 0x1322f70 2024-03-27T08:43:33.727 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 -- 172.21.15.104:0/1149382479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 msgr2=0x13b4d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.727 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 --2- 172.21.15.104:0/1149382479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4d20 crc :-1 s=READY pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.727 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 -- 172.21.15.104:0/1149382479 shutdown_connections 2024-03-27T08:43:33.727 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 --2- 172.21.15.104:0/1149382479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4d20 unknown :-1 s=CLOSED pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.727 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 -- 172.21.15.104:0/1149382479 >> 172.21.15.104:0/1149382479 conn(0x123cea0 msgr2=0x1325d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.727 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 -- 172.21.15.104:0/1149382479 shutdown_connections 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.730+0000 7f5860833ec0 1 -- 172.21.15.104:0/1149382479 wait complete. 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.731+0000 7f5860833ec0 1 Processor -- start 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.731+0000 7f5860833ec0 1 -- start start 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f5860833ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f5860833ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1285b30 con 0x1322f70 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f585899b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f585899b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34912/0 (socket says 172.21.15.104:34912) 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f585899b640 1 -- 172.21.15.104:0/3467146056 learned_addr learned my addr 172.21.15.104:0/3467146056 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f585899b640 1 -- 172.21.15.104:0/3467146056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x121ae30 con 0x1322f70 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f585899b640 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4a60 crc :-1 s=READY pgs=12963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f58517fa640 1 -- 172.21.15.104:0/3467146056 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5844033cb0 con 0x1322f70 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f58517fa640 1 -- 172.21.15.104:0/3467146056 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5844033e10 con 0x1322f70 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f58517fa640 1 -- 172.21.15.104:0/3467146056 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5844034570 con 0x1322f70 2024-03-27T08:43:33.728 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.732+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x121a890 con 0x1322f70 2024-03-27T08:43:33.729 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.733+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x13b5730 con 0x1322f70 2024-03-27T08:43:33.729 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.733+0000 7f58517fa640 1 -- 172.21.15.104:0/3467146056 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f5844031040 con 0x1322f70 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.734+0000 7f58517fa640 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5840037030 0x7f5840039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.734+0000 7f58517fa640 1 -- 172.21.15.104:0/3467146056 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(704..704 src has 397..704) ==== 4179+0+0 (crc 0 0 0) 0x7f584402e040 con 0x1322f70 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.734+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> v2:172.21.15.104:6801/1224298329 conn(0x13238c0 0x144d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.734+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 --> v2:172.21.15.104:6801/1224298329 -- osd_op(unknown.0.0:1 201.b 201:d1337354:::obj10:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e704) -- 0x13a1bd0 con 0x13238c0 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.734+0000 7f58537fe640 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5840037030 0x7f5840039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.734+0000 7f58537fe640 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5840037030 0x7f5840039540 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.730 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.735+0000 7f585919c640 1 --2- 172.21.15.104:0/3467146056 >> v2:172.21.15.104:6801/1224298329 conn(0x13238c0 0x144d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:43:33.731 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.737+0000 7f585919c640 1 --2- 172.21.15.104:0/3467146056 >> v2:172.21.15.104:6801/1224298329 conn(0x13238c0 0x144d4b0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:43:33.743 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.748+0000 7f585919c640 1 -- 172.21.15.104:0/3467146056 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== osd_op_reply(1 obj10 [writefull 0~11812] v704'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f584c00c430 con 0x13238c0 2024-03-27T08:43:33.743 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.748+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 >> v2:172.21.15.104:6801/1224298329 conn(0x13238c0 msgr2=0x144d4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.743 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.748+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> v2:172.21.15.104:6801/1224298329 conn(0x13238c0 0x144d4b0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.743 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5840037030 msgr2=0x7f5840039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.743 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5840037030 0x7f5840039540 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.743 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 msgr2=0x13b4a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4a60 crc :-1 s=READY pgs=12963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 shutdown_connections 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5840037030 0x7f5840039540 unknown :-1 s=CLOSED pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> v2:172.21.15.104:6801/1224298329 conn(0x13238c0 0x144d4b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 --2- 172.21.15.104:0/3467146056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1322f70 0x13b4a60 unknown :-1 s=CLOSED pgs=12963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 >> 172.21.15.104:0/3467146056 conn(0x123cea0 msgr2=0x13baf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 shutdown_connections 2024-03-27T08:43:33.744 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:43:33.749+0000 7f5860833ec0 1 -- 172.21.15.104:0/3467146056 wait complete. 2024-03-27T08:43:33.745 INFO:tasks.workunit.client.0.smithi104.stderr:+ sleep 30 2024-03-27T08:44:03.747 INFO:tasks.workunit.client.0.smithi104.stderr:+ rados -p 20a90cab-b0dc-43af-9f0a-bd39527c62c7 put threemore /etc/passwd 2024-03-27T08:44:03.789 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.795+0000 7f4a43ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:03.794 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.800+0000 7f4a43ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:03.794 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.800+0000 7f4a43ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:03.795 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.801+0000 7f4a43ec9ec0 1 Processor -- start 2024-03-27T08:44:03.795 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.801+0000 7f4a43ec9ec0 1 -- start start 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a43ec9ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1fd52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a43ec9ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1eae0b0 con 0x1fda360 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a377fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1fd52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a377fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1fd52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:54272/0 (socket says 172.21.15.104:54272) 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a377fe640 1 -- 172.21.15.104:0/655386886 learned_addr learned my addr 172.21.15.104:0/655386886 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a377fe640 1 -- 172.21.15.104:0/655386886 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1e4e1e0 con 0x1fda360 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a377fe640 1 --2- 172.21.15.104:0/655386886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1fd52c0 crc :-1 s=READY pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56a18c4bb5bc5d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a367fc640 1 -- 172.21.15.104:0/655386886 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a2800b5d0 con 0x1fda360 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a367fc640 1 -- 172.21.15.104:0/655386886 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4a2800b730 con 0x1fda360 2024-03-27T08:44:03.797 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.802+0000 7f4a367fc640 1 -- 172.21.15.104:0/655386886 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a2800b9d0 con 0x1fda360 2024-03-27T08:44:03.798 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/655386886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 msgr2=0x1fd52c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/655386886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1fd52c0 crc :-1 s=READY pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/655386886 shutdown_connections 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/655386886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1fd52c0 unknown :-1 s=CLOSED pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/655386886 >> 172.21.15.104:0/655386886 conn(0x1e62ea0 msgr2=0x1f4c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/655386886 shutdown_connections 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.803+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/655386886 wait complete. 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.804+0000 7f4a43ec9ec0 1 Processor -- start 2024-03-27T08:44:03.799 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.804+0000 7f4a43ec9ec0 1 -- start start 2024-03-27T08:44:03.801 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.805+0000 7f4a43ec9ec0 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1f416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.805+0000 7f4a43ec9ec0 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x1eabb30 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.805+0000 7f4a377fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1f416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.805+0000 7f4a377fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1f416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:54280/0 (socket says 172.21.15.104:54280) 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.805+0000 7f4a377fe640 1 -- 172.21.15.104:0/1770277469 learned_addr learned my addr 172.21.15.104:0/1770277469 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a377fe640 1 -- 172.21.15.104:0/1770277469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1e40e30 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a377fe640 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1f416f0 crc :-1 s=READY pgs=12967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a357fa640 1 -- 172.21.15.104:0/1770277469 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a28012910 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a357fa640 1 -- 172.21.15.104:0/1770277469 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4a28005040 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a357fa640 1 -- 172.21.15.104:0/1770277469 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a28012ba0 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1e40890 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.806+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1fd55f0 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.807+0000 7f4a357fa640 1 -- 172.21.15.104:0/1770277469 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f4a2800d040 con 0x1fda360 2024-03-27T08:44:03.802 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.807+0000 7f4a357fa640 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f4a18037030 0x7f4a18039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.807+0000 7f4a357fa640 1 -- 172.21.15.104:0/1770277469 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(706..706 src has 397..706) ==== 4179+0+0 (crc 0 0 0) 0x7f4a2800a040 con 0x1fda360 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.807+0000 7f4a357fa640 1 -- 172.21.15.104:0/1770277469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=707}) -- 0x7f4a1803da40 con 0x1fda360 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.807+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> v2:172.21.15.177:6805/2552358937 conn(0x1fdd190 0x1f43170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.807+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 --> v2:172.21.15.177:6805/2552358937 -- osd_op(unknown.0.0:1 200.2 200:47cad802:::threemore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x1fd59b0 con 0x1fdd190 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.808+0000 7f4a2ffff640 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f4a18037030 0x7f4a18039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.808+0000 7f4a2ffff640 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f4a18037030 0x7f4a18039540 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.803 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.809+0000 7f4a3cb9c640 1 --2- 172.21.15.104:0/1770277469 >> v2:172.21.15.177:6805/2552358937 conn(0x1fdd190 0x1f43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:44:03.807 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.813+0000 7f4a3cb9c640 1 --2- 172.21.15.104:0/1770277469 >> v2:172.21.15.177:6805/2552358937 conn(0x1fdd190 0x1f43170 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a3cb9c640 1 -- 172.21.15.104:0/1770277469 <== osd.2 v2:172.21.15.177:6805/2552358937 1 ==== osd_op_reply(1 threemore [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f4a3000a430 con 0x1fdd190 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 >> v2:172.21.15.177:6805/2552358937 conn(0x1fdd190 msgr2=0x1f43170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> v2:172.21.15.177:6805/2552358937 conn(0x1fdd190 0x1f43170 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f4a18037030 msgr2=0x7f4a18039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f4a18037030 0x7f4a18039540 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 msgr2=0x1f416f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1f416f0 crc :-1 s=READY pgs=12967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.821 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 shutdown_connections 2024-03-27T08:44:03.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> v2:172.21.15.177:6805/2552358937 conn(0x1fdd190 0x1f43170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f4a18037030 0x7f4a18039540 unknown :-1 s=CLOSED pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 --2- 172.21.15.104:0/1770277469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x1fda360 0x1f416f0 unknown :-1 s=CLOSED pgs=12967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 >> 172.21.15.104:0/1770277469 conn(0x1e62ea0 msgr2=0x1fda770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:03.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 shutdown_connections 2024-03-27T08:44:03.822 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.826+0000 7f4a43ec9ec0 1 -- 172.21.15.104:0/1770277469 wait complete. 2024-03-27T08:44:03.823 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_bytes 0 2024-03-27T08:44:03.917 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.915+0000 7ff3ad26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:03.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.924+0000 7ff3ad26d640 1 Processor -- start 2024-03-27T08:44:03.918 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.924+0000 7ff3ad26d640 1 -- start start 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.925+0000 7ff3ad26d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff3a809f860 0x7ff3a809fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.925+0000 7ff3ad26d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a806d980 con 0x7ff3a8134770 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.925+0000 7ff3ad26d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a806dc20 con 0x7ff3a809f860 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.925+0000 7ff3a6ffd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff3a809f860 0x7ff3a809fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a67fc640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff3a8134770 0x7ff3a8134b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:51126/0 (socket says 172.21.15.104:51126) 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a6ffd640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff3a809f860 0x7ff3a809fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:54282/0 (socket says 172.21.15.104:54282) 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a67fc640 1 -- 172.21.15.104:0/2122302480 learned_addr learned my addr 172.21.15.104:0/2122302480 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a6ffd640 1 -- 172.21.15.104:0/2122302480 >> v1:172.21.15.104:6789/0 conn(0x7ff3a8134770 legacy=0x7ff3a8134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a6ffd640 1 -- 172.21.15.104:0/2122302480 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a809f430 con 0x7ff3a809f860 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a6ffd640 1 --2- 172.21.15.104:0/2122302480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a809fc70 crc :-1 s=READY pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed608564022c8088 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.921 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a5ffb640 1 -- 172.21.15.104:0/2122302480 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff39800e040 con 0x7ff3a809f860 2024-03-27T08:44:03.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a5ffb640 1 -- 172.21.15.104:0/2122302480 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff398007670 con 0x7ff3a809f860 2024-03-27T08:44:03.922 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.926+0000 7ff3a5ffb640 1 -- 172.21.15.104:0/2122302480 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff398007910 con 0x7ff3a809f860 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 -- 172.21.15.104:0/2122302480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 msgr2=0x7ff3a809fc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 --2- 172.21.15.104:0/2122302480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a809fc70 crc :-1 s=READY pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 -- 172.21.15.104:0/2122302480 shutdown_connections 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 --2- 172.21.15.104:0/2122302480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a809fc70 unknown :-1 s=CLOSED pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 -- 172.21.15.104:0/2122302480 >> 172.21.15.104:0/2122302480 conn(0x7ff3a80a4bc0 msgr2=0x7ff3a80a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 -- 172.21.15.104:0/2122302480 shutdown_connections 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 1 -- 172.21.15.104:0/2122302480 wait complete. 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.927+0000 7ff3ad26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.928+0000 7ff3ad26d640 1 Processor -- start 2024-03-27T08:44:03.923 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.929+0000 7ff3ad26d640 1 -- start start 2024-03-27T08:44:03.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.929+0000 7ff3ad26d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a811e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.925 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3ad26d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81205e0 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3a6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a811e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3a6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a811e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:54298/0 (socket says 172.21.15.104:54298) 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3a6ffd640 1 -- 172.21.15.104:0/4094136370 learned_addr learned my addr 172.21.15.104:0/4094136370 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3a6ffd640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8120b70 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3a6ffd640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a811e360 crc :-1 s=READY pgs=12970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff39800e040 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff398012a80 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a8121c30 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff398012d20 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.930+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a81220b0 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.931+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff3980153f0 con 0x7ff3a809f860 2024-03-27T08:44:03.926 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.931+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a80693c0 con 0x7ff3a809f860 2024-03-27T08:44:03.927 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.931+0000 7ff38f7fe640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff3940387a0 0x7ff39403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:03.927 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.931+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(706..706 src has 397..706) ==== 4179+0+0 (crc 0 0 0) 0x7ff398049de0 con 0x7ff3a809f860 2024-03-27T08:44:03.927 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.931+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=707}) -- 0x7ff39403f150 con 0x7ff3a809f860 2024-03-27T08:44:03.927 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.932+0000 7ff3a67fc640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff3940387a0 0x7ff39403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:03.927 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.932+0000 7ff3a67fc640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff3940387a0 0x7ff39403acb0 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:03.930 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:03.936+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff39800e1a0 con 0x7ff3a809f860 2024-03-27T08:44:04.108 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:04.113+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"} v 0) -- 0x7ff3a8068d40 con 0x7ff3a809f860 2024-03-27T08:44:05.035 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:05.040+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v707) ==== 217+0+0 (crc 0 0 0) 0x7ff398010280 con 0x7ff3a809f860 2024-03-27T08:44:05.035 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:05.040+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(707..707 src has 397..707) ==== 628+0+0 (crc 0 0 0) 0x7ff398012e80 con 0x7ff3a809f860 2024-03-27T08:44:05.035 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:05.041+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=708}) -- 0x7ff39403ffc0 con 0x7ff3a809f860 2024-03-27T08:44:05.140 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:05.146+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"} v 0) -- 0x7ff3a8120880 con 0x7ff3a809f860 2024-03-27T08:44:06.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.041+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v708) ==== 217+0+0 (crc 0 0 0) 0x7ff39804b040 con 0x7ff3a809f860 2024-03-27T08:44:06.036 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_bytes = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:44:06.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.041+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 <== mon.0 v2:172.21.15.104:3300/0 10 ==== osd_map(708..708 src has 397..708) ==== 628+0+0 (crc 0 0 0) 0x7ff398010440 con 0x7ff3a809f860 2024-03-27T08:44:06.036 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.041+0000 7ff38f7fe640 1 -- 172.21.15.104:0/4094136370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=709}) -- 0x7ff394040230 con 0x7ff3a809f860 2024-03-27T08:44:06.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff3940387a0 msgr2=0x7ff39403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:06.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff3940387a0 0x7ff39403acb0 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:06.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 msgr2=0x7ff3a811e360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:06.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a811e360 crc :-1 s=READY pgs=12970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:06.038 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 shutdown_connections 2024-03-27T08:44:06.039 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff3940387a0 0x7ff39403acb0 unknown :-1 s=CLOSED pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:06.039 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 --2- 172.21.15.104:0/4094136370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3a809f860 0x7ff3a811e360 unknown :-1 s=CLOSED pgs=12970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:06.039 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 >> 172.21.15.104:0/4094136370 conn(0x7ff3a80a4bc0 msgr2=0x7ff3a8133000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:06.039 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 shutdown_connections 2024-03-27T08:44:06.039 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.043+0000 7ff3ad26d640 1 -- 172.21.15.104:0/4094136370 wait complete. 2024-03-27T08:44:06.047 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool set-quota 20a90cab-b0dc-43af-9f0a-bd39527c62c7 max_objects 0 2024-03-27T08:44:06.139 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.137+0000 7f5d63e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:06.140 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.146+0000 7f5d63e6d640 1 Processor -- start 2024-03-27T08:44:06.140 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.146+0000 7f5d63e6d640 1 -- start start 2024-03-27T08:44:06.142 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.147+0000 7f5d63e6d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5d5c131bf0 0x7f5d5c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.147+0000 7f5d63e6d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d5c06f7b0 con 0x7f5d5c06e350 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.147+0000 7f5d63e6d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5c06fa90 con 0x7f5d5c131bf0 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d61408640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f5d5c131bf0 0x7f5d5c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d61c09640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f5d5c06e350 0x7f5d5c1314e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:38198/0 (socket says 172.21.15.104:38198) 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d61c09640 1 -- 172.21.15.104:0/538302709 learned_addr learned my addr 172.21.15.104:0/538302709 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d61408640 1 -- 172.21.15.104:0/538302709 >> v1:172.21.15.104:6789/0 conn(0x7f5d5c06e350 legacy=0x7f5d5c1314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d61408640 1 -- 172.21.15.104:0/538302709 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c06f4a0 con 0x7f5d5c131bf0 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d61408640 1 --2- 172.21.15.104:0/538302709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c131bf0 0x7f5d5c136a10 crc :-1 s=READY pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adae970b8bc54b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d60c07640 1 -- 172.21.15.104:0/538302709 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d4c009d90 con 0x7f5d5c131bf0 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d60c07640 1 -- 172.21.15.104:0/538302709 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5d4c00e230 con 0x7f5d5c131bf0 2024-03-27T08:44:06.143 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.148+0000 7f5d60c07640 1 -- 172.21.15.104:0/538302709 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d4c00e4d0 con 0x7f5d5c131bf0 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 -- 172.21.15.104:0/538302709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c131bf0 msgr2=0x7f5d5c136a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 --2- 172.21.15.104:0/538302709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c131bf0 0x7f5d5c136a10 crc :-1 s=READY pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 -- 172.21.15.104:0/538302709 shutdown_connections 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 --2- 172.21.15.104:0/538302709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c131bf0 0x7f5d5c136a10 unknown :-1 s=CLOSED pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 -- 172.21.15.104:0/538302709 >> 172.21.15.104:0/538302709 conn(0x7f5d5c0a0b50 msgr2=0x7f5d5c0a0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 -- 172.21.15.104:0/538302709 shutdown_connections 2024-03-27T08:44:06.144 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 1 -- 172.21.15.104:0/538302709 wait complete. 2024-03-27T08:44:06.145 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.149+0000 7f5d63e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:06.145 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.150+0000 7f5d63e6d640 1 Processor -- start 2024-03-27T08:44:06.145 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.150+0000 7f5d63e6d640 1 -- start start 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.151+0000 7f5d63e6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 0x7f5d5c11cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.151+0000 7f5d63e6d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c11f000 con 0x7f5d5c06e350 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d61c09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 0x7f5d5c11cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d61c09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 0x7f5d5c11cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49460/0 (socket says 172.21.15.104:49460) 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d61c09640 1 -- 172.21.15.104:0/154525007 learned_addr learned my addr 172.21.15.104:0/154525007 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d61c09640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c11e300 con 0x7f5d5c06e350 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d61c09640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 0x7f5d5c11cf40 crc :-1 s=READY pgs=12973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d58005a60 con 0x7f5d5c06e350 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5d58005bc0 con 0x7f5d5c06e350 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c121020 con 0x7f5d5c06e350 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d58005e60 con 0x7f5d5c06e350 2024-03-27T08:44:06.147 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.152+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c11dff0 con 0x7f5d5c06e350 2024-03-27T08:44:06.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.153+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c0693c0 con 0x7f5d5c06e350 2024-03-27T08:44:06.148 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.153+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f5d5800c780 con 0x7f5d5c06e350 2024-03-27T08:44:06.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.153+0000 7f5d527fc640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5d400387a0 0x7f5d4003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:06.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.153+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(708..708 src has 397..708) ==== 4179+0+0 (crc 0 0 0) 0x7f5d58048520 con 0x7f5d5c06e350 2024-03-27T08:44:06.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.153+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=709}) -- 0x7f5d4003f150 con 0x7f5d5c06e350 2024-03-27T08:44:06.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.154+0000 7f5d61408640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5d400387a0 0x7f5d4003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:06.149 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.154+0000 7f5d61408640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5d400387a0 0x7f5d4003acb0 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:06.153 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.159+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f5d5800d330 con 0x7f5d5c06e350 2024-03-27T08:44:06.330 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:06.336+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"} v 0) -- 0x7f5d5c075190 con 0x7f5d5c06e350 2024-03-27T08:44:07.042 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:07.047+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v709) ==== 221+0+0 (crc 0 0 0) 0x7f5d5800d150 con 0x7f5d5c06e350 2024-03-27T08:44:07.042 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:07.047+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(709..709 src has 397..709) ==== 628+0+0 (crc 0 0 0) 0x7f5d5800d510 con 0x7f5d5c06e350 2024-03-27T08:44:07.042 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:07.048+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=710}) -- 0x7f5d4003ffc0 con 0x7f5d5c06e350 2024-03-27T08:44:07.198 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:07.204+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"} v 0) -- 0x7f5d5c120c60 con 0x7f5d5c06e350 2024-03-27T08:44:08.046 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.051+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 v710) ==== 221+0+0 (crc 0 0 0) 0x7f5d5800d720 con 0x7f5d5c06e350 2024-03-27T08:44:08.046 INFO:tasks.workunit.client.0.smithi104.stderr:set-quota max_objects = 0 for pool 20a90cab-b0dc-43af-9f0a-bd39527c62c7 2024-03-27T08:44:08.046 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.051+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 <== mon.0 v2:172.21.15.104:3300/0 10 ==== osd_map(710..710 src has 397..710) ==== 628+0+0 (crc 0 0 0) 0x7f5d5800d720 con 0x7f5d5c06e350 2024-03-27T08:44:08.046 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.051+0000 7f5d527fc640 1 -- 172.21.15.104:0/154525007 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=711}) -- 0x7f5d40040230 con 0x7f5d5c06e350 2024-03-27T08:44:08.048 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.053+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5d400387a0 msgr2=0x7f5d4003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:08.048 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.053+0000 7f5d63e6d640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5d400387a0 0x7f5d4003acb0 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:08.048 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 msgr2=0x7f5d5c11cf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:08.048 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 0x7f5d5c11cf40 crc :-1 s=READY pgs=12973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:08.048 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 shutdown_connections 2024-03-27T08:44:08.048 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f5d400387a0 0x7f5d4003acb0 unknown :-1 s=CLOSED pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:08.049 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 --2- 172.21.15.104:0/154525007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5c06e350 0x7f5d5c11cf40 unknown :-1 s=CLOSED pgs=12973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:08.049 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 >> 172.21.15.104:0/154525007 conn(0x7f5d5c0a0b50 msgr2=0x7f5d5c06fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:08.049 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 shutdown_connections 2024-03-27T08:44:08.049 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:08.054+0000 7f5d63e6d640 1 -- 172.21.15.104:0/154525007 wait complete. 2024-03-27T08:44:08.058 INFO:tasks.workunit.client.0.smithi104.stderr:+ sleep 30 2024-03-27T08:44:38.060 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool delete 20a90cab-b0dc-43af-9f0a-bd39527c62c7 20a90cab-b0dc-43af-9f0a-bd39527c62c7 --yes-i-really-really-mean-it 2024-03-27T08:44:38.162 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.159+0000 7fcc74d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:38.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.168+0000 7fcc74d9d640 1 Processor -- start 2024-03-27T08:44:38.163 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.168+0000 7fcc74d9d640 1 -- start start 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.169+0000 7fcc74d9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcc70134970 0x7fcc70136d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.169+0000 7fcc74d9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7012e160 con 0x7fcc7012e6a0 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.169+0000 7fcc74d9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7006c750 con 0x7fcc70134970 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6dd9b640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcc70134970 0x7fcc70136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6e59c640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fcc7012e6a0 0x7fcc7012eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43162/0 (socket says 172.21.15.104:43162) 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6e59c640 1 -- 172.21.15.104:0/1502813741 learned_addr learned my addr 172.21.15.104:0/1502813741 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6dd9b640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fcc70134970 0x7fcc70136d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42216/0 (socket says 172.21.15.104:42216) 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6dd9b640 1 -- 172.21.15.104:0/1502813741 >> v1:172.21.15.104:6789/0 conn(0x7fcc7012e6a0 legacy=0x7fcc7012eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6dd9b640 1 -- 172.21.15.104:0/1502813741 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7006ae80 con 0x7fcc70134970 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6dd9b640 1 --2- 172.21.15.104:0/1502813741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc70134970 0x7fcc70136d90 crc :-1 s=READY pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125ed12dd47355cb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:38.165 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6d59a640 1 -- 172.21.15.104:0/1502813741 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcc64009d90 con 0x7fcc70134970 2024-03-27T08:44:38.166 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6d59a640 1 -- 172.21.15.104:0/1502813741 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcc6400e230 con 0x7fcc70134970 2024-03-27T08:44:38.166 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.170+0000 7fcc6d59a640 1 -- 172.21.15.104:0/1502813741 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcc6400e4d0 con 0x7fcc70134970 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 -- 172.21.15.104:0/1502813741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc70134970 msgr2=0x7fcc70136d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 --2- 172.21.15.104:0/1502813741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc70134970 0x7fcc70136d90 crc :-1 s=READY pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 -- 172.21.15.104:0/1502813741 shutdown_connections 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 --2- 172.21.15.104:0/1502813741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc70134970 0x7fcc70136d90 unknown :-1 s=CLOSED pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 -- 172.21.15.104:0/1502813741 >> 172.21.15.104:0/1502813741 conn(0x7fcc7009fad0 msgr2=0x7fcc7009fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 -- 172.21.15.104:0/1502813741 shutdown_connections 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 1 -- 172.21.15.104:0/1502813741 wait complete. 2024-03-27T08:44:38.167 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.171+0000 7fcc74d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:38.168 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.173+0000 7fcc74d9d640 1 Processor -- start 2024-03-27T08:44:38.168 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.173+0000 7fcc74d9d640 1 -- start start 2024-03-27T08:44:38.169 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc74d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 0x7fcc7012a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:38.169 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc74d9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7012d6a0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.169 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc6e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 0x7fcc7012a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:38.169 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc6e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 0x7fcc7012a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42232/0 (socket says 172.21.15.104:42232) 2024-03-27T08:44:38.169 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc6e59c640 1 -- 172.21.15.104:0/3603887658 learned_addr learned my addr 172.21.15.104:0/3603887658 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:38.169 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc6e59c640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7012bd00 con 0x7fcc7012e6a0 2024-03-27T08:44:38.170 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.174+0000 7fcc6e59c640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 0x7fcc7012a170 crc :-1 s=READY pgs=12976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:38.170 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.175+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcc600070b0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.170 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.175+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcc60007210 con 0x7fcc7012e6a0 2024-03-27T08:44:38.170 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.175+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcc600074b0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.170 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.175+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7012d390 con 0x7fcc7012e6a0 2024-03-27T08:44:38.170 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.175+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7012ddc0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.171 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.176+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fcc600076a0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.171 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.176+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc700693c0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.171 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.176+0000 7fcc5effd640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fcc500387a0 0x7fcc5003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:38.171 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.176+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(710..710 src has 397..710) ==== 4179+0+0 (crc 0 0 0) 0x7fcc60048ee0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.171 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.176+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=711}) -- 0x7fcc5003f150 con 0x7fcc7012e6a0 2024-03-27T08:44:38.171 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.176+0000 7fcc6dd9b640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fcc500387a0 0x7fcc5003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:38.172 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.177+0000 7fcc6dd9b640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fcc500387a0 0x7fcc5003acb0 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:38.176 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.181+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fcc600094a0 con 0x7fcc7012e6a0 2024-03-27T08:44:38.355 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:38.360+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pool2": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcc7012c5a0 con 0x7fcc7012e6a0 2024-03-27T08:44:39.080 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.085+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pool2": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "yes_i_really_really_mean_it": true}]=0 pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' removed v711) ==== 248+0+0 (crc 0 0 0) 0x7fcc600090b0 con 0x7fcc7012e6a0 2024-03-27T08:44:39.081 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.086+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(711..711 src has 397..711) ==== 296+0+0 (crc 0 0 0) 0x7fcc60009290 con 0x7fcc7012e6a0 2024-03-27T08:44:39.081 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.086+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=712}) -- 0x7fcc5003fcb0 con 0x7fcc7012e6a0 2024-03-27T08:44:39.184 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.190+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pool2": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcc700008d0 con 0x7fcc7012e6a0 2024-03-27T08:44:39.185 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.190+0000 7fcc5effd640 1 -- 172.21.15.104:0/3603887658 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "pool2": "20a90cab-b0dc-43af-9f0a-bd39527c62c7", "yes_i_really_really_mean_it": true}]=0 pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' does not exist v711) ==== 255+0+0 (crc 0 0 0) 0x7fcc60007950 con 0x7fcc7012e6a0 2024-03-27T08:44:39.185 INFO:tasks.workunit.client.0.smithi104.stderr:pool '20a90cab-b0dc-43af-9f0a-bd39527c62c7' does not exist 2024-03-27T08:44:39.187 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fcc500387a0 msgr2=0x7fcc5003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:39.187 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fcc500387a0 0x7fcc5003acb0 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:39.187 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 msgr2=0x7fcc7012a170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 0x7fcc7012a170 crc :-1 s=READY pgs=12976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 shutdown_connections 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fcc500387a0 0x7fcc5003acb0 unknown :-1 s=CLOSED pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 --2- 172.21.15.104:0/3603887658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc7012e6a0 0x7fcc7012a170 unknown :-1 s=CLOSED pgs=12976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 >> 172.21.15.104:0/3603887658 conn(0x7fcc7009fad0 msgr2=0x7fcc70132ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 shutdown_connections 2024-03-27T08:44:39.188 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.192+0000 7fcc74d9d640 1 -- 172.21.15.104:0/3603887658 wait complete. 2024-03-27T08:44:39.197 INFO:tasks.workunit.client.0.smithi104.stderr:+ ceph osd pool delete b94fed44-a57c-48b9-b4e4-7a1f5935fb7f b94fed44-a57c-48b9-b4e4-7a1f5935fb7f --yes-i-really-really-mean-it 2024-03-27T08:44:39.289 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.286+0000 7f14b446d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:39.290 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.295+0000 7f14b446d640 1 Processor -- start 2024-03-27T08:44:39.290 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.295+0000 7f14b446d640 1 -- start start 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.296+0000 7f14b446d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14ac132c40 0x7f14ac133050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.296+0000 7f14b446d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ac071e50 con 0x7f14ac070c30 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.296+0000 7f14b446d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f14ac132810 con 0x7f14ac132c40 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.296+0000 7f14b2209640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14ac132c40 0x7f14ac133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.296+0000 7f14b2209640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f14ac132c40 0x7f14ac133050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42242/0 (socket says 172.21.15.104:42242) 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.296+0000 7f14b1a08640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f14ac070c30 0x7f14ac071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43164/0 (socket says 172.21.15.104:43164) 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b2209640 1 -- 172.21.15.104:0/338057143 learned_addr learned my addr 172.21.15.104:0/338057143 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b2209640 1 -- 172.21.15.104:0/338057143 >> v1:172.21.15.104:6789/0 conn(0x7f14ac070c30 legacy=0x7f14ac071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b2209640 1 -- 172.21.15.104:0/338057143 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac06ae80 con 0x7f14ac132c40 2024-03-27T08:44:39.292 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b2209640 1 --2- 172.21.15.104:0/338057143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac132c40 0x7f14ac133050 crc :-1 s=READY pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938dff7250c5d17f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:39.293 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b1207640 1 -- 172.21.15.104:0/338057143 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f14a8009e50 con 0x7f14ac132c40 2024-03-27T08:44:39.293 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b1207640 1 -- 172.21.15.104:0/338057143 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f14a800e230 con 0x7f14ac132c40 2024-03-27T08:44:39.293 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.297+0000 7f14b1207640 1 -- 172.21.15.104:0/338057143 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f14a800e4d0 con 0x7f14ac132c40 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 -- 172.21.15.104:0/338057143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac132c40 msgr2=0x7f14ac133050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 --2- 172.21.15.104:0/338057143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac132c40 0x7f14ac133050 crc :-1 s=READY pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 -- 172.21.15.104:0/338057143 shutdown_connections 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 --2- 172.21.15.104:0/338057143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac132c40 0x7f14ac133050 unknown :-1 s=CLOSED pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 -- 172.21.15.104:0/338057143 >> 172.21.15.104:0/338057143 conn(0x7f14ac09fad0 msgr2=0x7f14ac09fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 -- 172.21.15.104:0/338057143 shutdown_connections 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 1 -- 172.21.15.104:0/338057143 wait complete. 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.298+0000 7f14b446d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:39.294 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.299+0000 7f14b446d640 1 Processor -- start 2024-03-27T08:44:39.295 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.300+0000 7f14b446d640 1 -- start start 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.300+0000 7f14b446d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 0x7f14ac1c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.300+0000 7f14b446d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac12f720 con 0x7f14ac070c30 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b2209640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 0x7f14ac1c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b2209640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 0x7f14ac1c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42256/0 (socket says 172.21.15.104:42256) 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b2209640 1 -- 172.21.15.104:0/4033775124 learned_addr learned my addr 172.21.15.104:0/4033775124 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b2209640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac12e2e0 con 0x7f14ac070c30 2024-03-27T08:44:39.296 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b2209640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 0x7f14ac1c7a80 crc :-1 s=READY pgs=12979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:39.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f14a800ee00 con 0x7f14ac070c30 2024-03-27T08:44:39.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f14a8015420 con 0x7f14ac070c30 2024-03-27T08:44:39.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac12a370 con 0x7f14ac070c30 2024-03-27T08:44:39.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f14a80156c0 con 0x7f14ac070c30 2024-03-27T08:44:39.297 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.301+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac12fc20 con 0x7f14ac070c30 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.302+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f14a8015870 con 0x7f14ac070c30 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.302+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac0683d0 con 0x7f14ac070c30 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.302+0000 7f14a2ffd640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f14900387a0 0x7f149003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.302+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(711..711 src has 397..711) ==== 3791+0+0 (crc 0 0 0) 0x7f14a8049ad0 con 0x7f14ac070c30 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.302+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=712}) -- 0x7f149003ea10 con 0x7f14ac070c30 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.303+0000 7f14b1a08640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f14900387a0 0x7f149003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:39.298 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.303+0000 7f14b1a08640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f14900387a0 0x7f149003acb0 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:39.302 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.307+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f14ac0683d0 con 0x7f14ac070c30 2024-03-27T08:44:39.480 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:39.485+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pool2": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "yes_i_really_really_mean_it": true} v 0) -- 0x7f14ac0683d0 con 0x7f14ac070c30 2024-03-27T08:44:40.082 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.087+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pool2": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "yes_i_really_really_mean_it": true}]=0 pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' removed v712) ==== 248+0+0 (crc 0 0 0) 0x7f14ac0683d0 con 0x7f14ac070c30 2024-03-27T08:44:40.083 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.089+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 8 ==== osd_map(712..712 src has 397..712) ==== 296+0+0 (crc 0 0 0) 0x7f14a8012a20 con 0x7f14ac070c30 2024-03-27T08:44:40.084 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.089+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=713}) -- 0x7f149003f720 con 0x7f14ac070c30 2024-03-27T08:44:40.263 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.268+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pool2": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "yes_i_really_really_mean_it": true} v 0) -- 0x7f14ac0683d0 con 0x7f14ac070c30 2024-03-27T08:44:40.264 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.269+0000 7f14a2ffd640 1 -- 172.21.15.104:0/4033775124 <== mon.0 v2:172.21.15.104:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "pool2": "b94fed44-a57c-48b9-b4e4-7a1f5935fb7f", "yes_i_really_really_mean_it": true}]=0 pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' does not exist v712) ==== 255+0+0 (crc 0 0 0) 0x7f14ac0683d0 con 0x7f14ac070c30 2024-03-27T08:44:40.264 INFO:tasks.workunit.client.0.smithi104.stderr:pool 'b94fed44-a57c-48b9-b4e4-7a1f5935fb7f' does not exist 2024-03-27T08:44:40.267 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f14900387a0 msgr2=0x7f149003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f14900387a0 0x7f149003acb0 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 msgr2=0x7f14ac1c7a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 0x7f14ac1c7a80 crc :-1 s=READY pgs=12979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 shutdown_connections 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f14900387a0 0x7f149003acb0 unknown :-1 s=CLOSED pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 --2- 172.21.15.104:0/4033775124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ac070c30 0x7f14ac1c7a80 unknown :-1 s=CLOSED pgs=12979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.272+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 >> 172.21.15.104:0/4033775124 conn(0x7f14ac09fad0 msgr2=0x7f14ac09e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.273+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 shutdown_connections 2024-03-27T08:44:40.268 INFO:tasks.workunit.client.0.smithi104.stderr:2024-03-27T08:44:40.273+0000 7f14b446d640 1 -- 172.21.15.104:0/4033775124 wait complete. 2024-03-27T08:44:40.285 INFO:tasks.workunit.client.0.smithi104.stderr:+ echo OK 2024-03-27T08:44:40.286 INFO:tasks.workunit.client.0.smithi104.stdout:OK 2024-03-27T08:44:40.287 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-27T08:44:40.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-27T08:44:40.358 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-03-27T08:44:40.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-27T08:44:40.690 DEBUG:teuthology.parallel:result is None 2024-03-27T08:44:40.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T08:44:40.717 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T08:44:40.717 DEBUG:teuthology.orchestra.run.smithi104:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-27T08:44:40.783 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T08:44:40.783 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-27T08:44:40.794 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-27T08:44:40.794 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:44:40.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.935+0000 7ff22955d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:40.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.940+0000 7ff22955d640 1 Processor -- start 2024-03-27T08:44:40.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.940+0000 7ff22955d640 1 -- start start 2024-03-27T08:44:40.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff22955d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff224135830 0x7ff224137c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff22955d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2240732c0 con 0x7ff22412f5c0 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff22955d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7ff22409eeb0 con 0x7ff224135830 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff21bfff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff224135830 0x7ff224137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff222ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7ff22412f5c0 0x7ff22412f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43170/0 (socket says 172.21.15.104:43170) 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff222ffd640 1 -- 172.21.15.104:0/383254563 learned_addr learned my addr 172.21.15.104:0/383254563 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff21bfff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7ff224135830 0x7ff224137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42270/0 (socket says 172.21.15.104:42270) 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff21bfff640 1 -- 172.21.15.104:0/383254563 >> v1:172.21.15.104:6789/0 conn(0x7ff22412f5c0 legacy=0x7ff22412f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff21bfff640 1 -- 172.21.15.104:0/383254563 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2241300e0 con 0x7ff224135830 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.941+0000 7ff2227fc640 1 -- 172.21.15.104:0/383254563 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607174255 0 0) 0x7ff2240732c0 con 0x7ff22412f5c0 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.942+0000 7ff21bfff640 1 --2- 172.21.15.104:0/383254563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224135830 0x7ff224137c50 crc :-1 s=READY pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787269c5dd34a907 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.942+0000 7ff2227fc640 1 -- 172.21.15.104:0/383254563 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff20c009d90 con 0x7ff224135830 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.942+0000 7ff2227fc640 1 -- 172.21.15.104:0/383254563 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff20c00e230 con 0x7ff224135830 2024-03-27T08:44:40.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.942+0000 7ff2227fc640 1 -- 172.21.15.104:0/383254563 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff20c00e4d0 con 0x7ff224135830 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.942+0000 7ff22955d640 1 -- 172.21.15.104:0/383254563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224135830 msgr2=0x7ff224137c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.942+0000 7ff22955d640 1 --2- 172.21.15.104:0/383254563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224135830 0x7ff224137c50 crc :-1 s=READY pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.943+0000 7ff22955d640 1 -- 172.21.15.104:0/383254563 shutdown_connections 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.943+0000 7ff22955d640 1 --2- 172.21.15.104:0/383254563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224135830 0x7ff224137c50 unknown :-1 s=CLOSED pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.943+0000 7ff22955d640 1 -- 172.21.15.104:0/383254563 >> 172.21.15.104:0/383254563 conn(0x7ff2240a29e0 msgr2=0x7ff2240a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.943+0000 7ff22955d640 1 -- 172.21.15.104:0/383254563 shutdown_connections 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.943+0000 7ff22955d640 1 -- 172.21.15.104:0/383254563 wait complete. 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.943+0000 7ff22955d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.944+0000 7ff22955d640 1 Processor -- start 2024-03-27T08:44:40.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.944+0000 7ff22955d640 1 -- start start 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.945+0000 7ff22955d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 0x7ff2241c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.945+0000 7ff22955d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff22411fc00 con 0x7ff22412f5c0 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.945+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 0x7ff2241c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.945+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 0x7ff2241c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42276/0 (socket says 172.21.15.104:42276) 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.945+0000 7ff222ffd640 1 -- 172.21.15.104:0/2200525913 learned_addr learned my addr 172.21.15.104:0/2200525913 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.945+0000 7ff222ffd640 1 -- 172.21.15.104:0/2200525913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2241c3ac0 con 0x7ff22412f5c0 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.946+0000 7ff222ffd640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 0x7ff2241c4100 crc :-1 s=READY pgs=12982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.946+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff214004b80 con 0x7ff22412f5c0 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.946+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff214004ce0 con 0x7ff22412f5c0 2024-03-27T08:44:40.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.946+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2241c3d60 con 0x7ff22412f5c0 2024-03-27T08:44:40.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.946+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff214007120 con 0x7ff22412f5c0 2024-03-27T08:44:40.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.946+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22411df50 con 0x7ff22412f5c0 2024-03-27T08:44:40.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.947+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2241c3f50 con 0x7ff22412f5c0 2024-03-27T08:44:40.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.947+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff2140072d0 con 0x7ff22412f5c0 2024-03-27T08:44:40.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.947+0000 7ff21b7fe640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff2040387a0 0x7ff20403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:40.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.947+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 397..712) ==== 3403+0+0 (crc 0 0 0) 0x7ff2140483a0 con 0x7ff22412f5c0 2024-03-27T08:44:40.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.947+0000 7ff21bfff640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff2040387a0 0x7ff20403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:40.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.948+0000 7ff21bfff640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff2040387a0 0x7ff20403acb0 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:40.947 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:40.952+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff21400a9e0 con 0x7ff22412f5c0 2024-03-27T08:44:41.120 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.126+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2240a1440 con 0x7ff2040387a0 2024-03-27T08:44:41.124 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.129+0000 7ff21b7fe640 1 -- 172.21.15.104:0/2200525913 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7ff2240a1440 con 0x7ff2040387a0 2024-03-27T08:44:41.125 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:44:41.126 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff2040387a0 msgr2=0x7ff20403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff2040387a0 0x7ff20403acb0 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 msgr2=0x7ff2241c4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 0x7ff2241c4100 crc :-1 s=READY pgs=12982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 shutdown_connections 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7ff2040387a0 0x7ff20403acb0 unknown :-1 s=CLOSED pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 --2- 172.21.15.104:0/2200525913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22412f5c0 0x7ff2241c4100 unknown :-1 s=CLOSED pgs=12982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 >> 172.21.15.104:0/2200525913 conn(0x7ff2240a29e0 msgr2=0x7ff2241324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 shutdown_connections 2024-03-27T08:44:41.128 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.132+0000 7ff22955d640 1 -- 172.21.15.104:0/2200525913 wait complete. 2024-03-27T08:44:41.138 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":1262,"stamp":"2024-03-27T08:44:40.476347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":296580,"kb_used_data":15268,"kb_used_omap":0,"kb_used_meta":281280,"kb_avail":419133820,"statfs":{"total":429496729600,"available":429193031680,"internally_reserved":0,"allocated":15634432,"data_stored":7008133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":288030720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002716"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.403955+0000","last_change":"2024-03-27T08:31:05.063650+0000","last_active":"2024-03-27T08:44:09.403955+0000","last_peered":"2024-03-27T08:44:09.403955+0000","last_clean":"2024-03-27T08:44:09.403955+0000","last_became_active":"2024-03-27T08:31:05.053480+0000","last_became_peered":"2024-03-27T08:31:05.053480+0000","last_unstale":"2024-03-27T08:44:09.403955+0000","last_undegraded":"2024-03-27T08:44:09.403955+0000","last_fullsized":"2024-03-27T08:44:09.403955+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.535366+0000","last_change":"2024-03-27T08:31:05.114356+0000","last_active":"2024-03-27T08:44:09.535366+0000","last_peered":"2024-03-27T08:44:09.535366+0000","last_clean":"2024-03-27T08:44:09.535366+0000","last_became_active":"2024-03-27T08:31:05.102560+0000","last_became_peered":"2024-03-27T08:31:05.102560+0000","last_unstale":"2024-03-27T08:44:09.535366+0000","last_undegraded":"2024-03-27T08:44:09.535366+0000","last_fullsized":"2024-03-27T08:44:09.535366+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.352281+0000","last_change":"2024-03-27T08:31:05.110354+0000","last_active":"2024-03-27T08:44:09.352281+0000","last_peered":"2024-03-27T08:44:09.352281+0000","last_clean":"2024-03-27T08:44:09.352281+0000","last_became_active":"2024-03-27T08:31:05.101799+0000","last_became_peered":"2024-03-27T08:31:05.101799+0000","last_unstale":"2024-03-27T08:44:09.352281+0000","last_undegraded":"2024-03-27T08:44:09.352281+0000","last_fullsized":"2024-03-27T08:44:09.352281+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.085002+0000","last_change":"2024-03-27T08:31:05.128758+0000","last_active":"2024-03-27T08:44:08.085002+0000","last_peered":"2024-03-27T08:44:08.085002+0000","last_clean":"2024-03-27T08:44:08.085002+0000","last_became_active":"2024-03-27T08:31:05.111183+0000","last_became_peered":"2024-03-27T08:31:05.111183+0000","last_unstale":"2024-03-27T08:44:08.085002+0000","last_undegraded":"2024-03-27T08:44:08.085002+0000","last_fullsized":"2024-03-27T08:44:08.085002+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.402434+0000","last_change":"2024-03-27T08:31:05.145624+0000","last_active":"2024-03-27T08:44:09.402434+0000","last_peered":"2024-03-27T08:44:09.402434+0000","last_clean":"2024-03-27T08:44:09.402434+0000","last_became_active":"2024-03-27T08:31:05.138042+0000","last_became_peered":"2024-03-27T08:31:05.138042+0000","last_unstale":"2024-03-27T08:44:09.402434+0000","last_undegraded":"2024-03-27T08:44:09.402434+0000","last_fullsized":"2024-03-27T08:44:09.402434+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.583640+0000","last_change":"2024-03-27T08:31:05.084677+0000","last_active":"2024-03-27T08:44:09.583640+0000","last_peered":"2024-03-27T08:44:09.583640+0000","last_clean":"2024-03-27T08:44:09.583640+0000","last_became_active":"2024-03-27T08:31:05.068818+0000","last_became_peered":"2024-03-27T08:31:05.068818+0000","last_unstale":"2024-03-27T08:44:09.583640+0000","last_undegraded":"2024-03-27T08:44:09.583640+0000","last_fullsized":"2024-03-27T08:44:09.583640+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.503546+0000","last_change":"2024-03-27T08:31:05.087048+0000","last_active":"2024-03-27T08:44:09.503546+0000","last_peered":"2024-03-27T08:44:09.503546+0000","last_clean":"2024-03-27T08:44:09.503546+0000","last_became_active":"2024-03-27T08:31:05.064801+0000","last_became_peered":"2024-03-27T08:31:05.064801+0000","last_unstale":"2024-03-27T08:44:09.503546+0000","last_undegraded":"2024-03-27T08:44:09.503546+0000","last_fullsized":"2024-03-27T08:44:09.503546+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.516989+0000","last_change":"2024-03-27T08:31:05.134669+0000","last_active":"2024-03-27T08:44:09.516989+0000","last_peered":"2024-03-27T08:44:09.516989+0000","last_clean":"2024-03-27T08:44:09.516989+0000","last_became_active":"2024-03-27T08:31:05.128790+0000","last_became_peered":"2024-03-27T08:31:05.128790+0000","last_unstale":"2024-03-27T08:44:09.516989+0000","last_undegraded":"2024-03-27T08:44:09.516989+0000","last_fullsized":"2024-03-27T08:44:09.516989+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.111329+0000","last_change":"2024-03-27T08:31:05.144335+0000","last_active":"2024-03-27T08:44:08.111329+0000","last_peered":"2024-03-27T08:44:08.111329+0000","last_clean":"2024-03-27T08:44:08.111329+0000","last_became_active":"2024-03-27T08:31:05.125515+0000","last_became_peered":"2024-03-27T08:31:05.125515+0000","last_unstale":"2024-03-27T08:44:08.111329+0000","last_undegraded":"2024-03-27T08:44:08.111329+0000","last_fullsized":"2024-03-27T08:44:08.111329+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.368034+0000","last_change":"2024-03-27T08:31:05.083033+0000","last_active":"2024-03-27T08:44:09.368034+0000","last_peered":"2024-03-27T08:44:09.368034+0000","last_clean":"2024-03-27T08:44:09.368034+0000","last_became_active":"2024-03-27T08:31:05.074459+0000","last_became_peered":"2024-03-27T08:31:05.074459+0000","last_unstale":"2024-03-27T08:44:09.368034+0000","last_undegraded":"2024-03-27T08:44:09.368034+0000","last_fullsized":"2024-03-27T08:44:09.368034+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.112194+0000","last_change":"2024-03-27T08:31:05.114956+0000","last_active":"2024-03-27T08:44:08.112194+0000","last_peered":"2024-03-27T08:44:08.112194+0000","last_clean":"2024-03-27T08:44:08.112194+0000","last_became_active":"2024-03-27T08:31:05.099066+0000","last_became_peered":"2024-03-27T08:31:05.099066+0000","last_unstale":"2024-03-27T08:44:08.112194+0000","last_undegraded":"2024-03-27T08:44:08.112194+0000","last_fullsized":"2024-03-27T08:44:08.112194+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.577379+0000","last_change":"2024-03-27T08:31:05.179980+0000","last_active":"2024-03-27T08:44:09.577379+0000","last_peered":"2024-03-27T08:44:09.577379+0000","last_clean":"2024-03-27T08:44:09.577379+0000","last_became_active":"2024-03-27T08:31:05.174215+0000","last_became_peered":"2024-03-27T08:31:05.174215+0000","last_unstale":"2024-03-27T08:44:09.577379+0000","last_undegraded":"2024-03-27T08:44:09.577379+0000","last_fullsized":"2024-03-27T08:44:09.577379+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.106888+0000","last_change":"2024-03-27T08:31:05.108652+0000","last_active":"2024-03-27T08:44:08.106888+0000","last_peered":"2024-03-27T08:44:08.106888+0000","last_clean":"2024-03-27T08:44:08.106888+0000","last_became_active":"2024-03-27T08:31:05.095788+0000","last_became_peered":"2024-03-27T08:31:05.095788+0000","last_unstale":"2024-03-27T08:44:08.106888+0000","last_undegraded":"2024-03-27T08:44:08.106888+0000","last_fullsized":"2024-03-27T08:44:08.106888+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.081647+0000","last_change":"2024-03-27T08:31:05.131740+0000","last_active":"2024-03-27T08:44:08.081647+0000","last_peered":"2024-03-27T08:44:08.081647+0000","last_clean":"2024-03-27T08:44:08.081647+0000","last_became_active":"2024-03-27T08:31:05.103731+0000","last_became_peered":"2024-03-27T08:31:05.103731+0000","last_unstale":"2024-03-27T08:44:08.081647+0000","last_undegraded":"2024-03-27T08:44:08.081647+0000","last_fullsized":"2024-03-27T08:44:08.081647+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.543849+0000","last_change":"2024-03-27T08:31:05.119311+0000","last_active":"2024-03-27T08:44:09.543849+0000","last_peered":"2024-03-27T08:44:09.543849+0000","last_clean":"2024-03-27T08:44:09.543849+0000","last_became_active":"2024-03-27T08:31:05.090562+0000","last_became_peered":"2024-03-27T08:31:05.090562+0000","last_unstale":"2024-03-27T08:44:09.543849+0000","last_undegraded":"2024-03-27T08:44:09.543849+0000","last_fullsized":"2024-03-27T08:44:09.543849+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.573731+0000","last_change":"2024-03-27T08:31:05.106623+0000","last_active":"2024-03-27T08:44:09.573731+0000","last_peered":"2024-03-27T08:44:09.573731+0000","last_clean":"2024-03-27T08:44:09.573731+0000","last_became_active":"2024-03-27T08:31:05.076510+0000","last_became_peered":"2024-03-27T08:31:05.076510+0000","last_unstale":"2024-03-27T08:44:09.573731+0000","last_undegraded":"2024-03-27T08:44:09.573731+0000","last_fullsized":"2024-03-27T08:44:09.573731+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.582170+0000","last_change":"2024-03-27T08:31:05.110196+0000","last_active":"2024-03-27T08:44:09.582170+0000","last_peered":"2024-03-27T08:44:09.582170+0000","last_clean":"2024-03-27T08:44:09.582170+0000","last_became_active":"2024-03-27T08:31:05.081587+0000","last_became_peered":"2024-03-27T08:31:05.081587+0000","last_unstale":"2024-03-27T08:44:09.582170+0000","last_undegraded":"2024-03-27T08:44:09.582170+0000","last_fullsized":"2024-03-27T08:44:09.582170+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.114526+0000","last_change":"2024-03-27T08:31:05.153706+0000","last_active":"2024-03-27T08:44:08.114526+0000","last_peered":"2024-03-27T08:44:08.114526+0000","last_clean":"2024-03-27T08:44:08.114526+0000","last_became_active":"2024-03-27T08:31:05.137462+0000","last_became_peered":"2024-03-27T08:31:05.137462+0000","last_unstale":"2024-03-27T08:44:08.114526+0000","last_undegraded":"2024-03-27T08:44:08.114526+0000","last_fullsized":"2024-03-27T08:44:08.114526+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.357491+0000","last_change":"2024-03-27T08:31:05.091071+0000","last_active":"2024-03-27T08:44:09.357491+0000","last_peered":"2024-03-27T08:44:09.357491+0000","last_clean":"2024-03-27T08:44:09.357491+0000","last_became_active":"2024-03-27T08:31:05.077928+0000","last_became_peered":"2024-03-27T08:31:05.077928+0000","last_unstale":"2024-03-27T08:44:09.357491+0000","last_undegraded":"2024-03-27T08:44:09.357491+0000","last_fullsized":"2024-03-27T08:44:09.357491+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.406084+0000","last_change":"2024-03-27T08:31:05.096352+0000","last_active":"2024-03-27T08:44:09.406084+0000","last_peered":"2024-03-27T08:44:09.406084+0000","last_clean":"2024-03-27T08:44:09.406084+0000","last_became_active":"2024-03-27T08:31:05.090906+0000","last_became_peered":"2024-03-27T08:31:05.090906+0000","last_unstale":"2024-03-27T08:44:09.406084+0000","last_undegraded":"2024-03-27T08:44:09.406084+0000","last_fullsized":"2024-03-27T08:44:09.406084+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.412305+0000","last_change":"2024-03-27T08:31:05.198445+0000","last_active":"2024-03-27T08:44:09.412305+0000","last_peered":"2024-03-27T08:44:09.412305+0000","last_clean":"2024-03-27T08:44:09.412305+0000","last_became_active":"2024-03-27T08:31:05.192295+0000","last_became_peered":"2024-03-27T08:31:05.192295+0000","last_unstale":"2024-03-27T08:44:09.412305+0000","last_undegraded":"2024-03-27T08:44:09.412305+0000","last_fullsized":"2024-03-27T08:44:09.412305+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.448126+0000","last_change":"2024-03-27T08:31:05.085798+0000","last_active":"2024-03-27T08:44:09.448126+0000","last_peered":"2024-03-27T08:44:09.448126+0000","last_clean":"2024-03-27T08:44:09.448126+0000","last_became_active":"2024-03-27T08:31:05.077394+0000","last_became_peered":"2024-03-27T08:31:05.077394+0000","last_unstale":"2024-03-27T08:44:09.448126+0000","last_undegraded":"2024-03-27T08:44:09.448126+0000","last_fullsized":"2024-03-27T08:44:09.448126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.347767+0000","last_change":"2024-03-27T08:31:05.120108+0000","last_active":"2024-03-27T08:44:09.347767+0000","last_peered":"2024-03-27T08:44:09.347767+0000","last_clean":"2024-03-27T08:44:09.347767+0000","last_became_active":"2024-03-27T08:31:05.114332+0000","last_became_peered":"2024-03-27T08:31:05.114332+0000","last_unstale":"2024-03-27T08:44:09.347767+0000","last_undegraded":"2024-03-27T08:44:09.347767+0000","last_fullsized":"2024-03-27T08:44:09.347767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.074802+0000","last_change":"2024-03-27T08:31:05.181415+0000","last_active":"2024-03-27T08:44:08.074802+0000","last_peered":"2024-03-27T08:44:08.074802+0000","last_clean":"2024-03-27T08:44:08.074802+0000","last_became_active":"2024-03-27T08:31:05.175030+0000","last_became_peered":"2024-03-27T08:31:05.175030+0000","last_unstale":"2024-03-27T08:44:08.074802+0000","last_undegraded":"2024-03-27T08:44:08.074802+0000","last_fullsized":"2024-03-27T08:44:08.074802+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.105795+0000","last_change":"2024-03-27T08:31:05.169539+0000","last_active":"2024-03-27T08:44:08.105795+0000","last_peered":"2024-03-27T08:44:08.105795+0000","last_clean":"2024-03-27T08:44:08.105795+0000","last_became_active":"2024-03-27T08:31:05.164396+0000","last_became_peered":"2024-03-27T08:31:05.164396+0000","last_unstale":"2024-03-27T08:44:08.105795+0000","last_undegraded":"2024-03-27T08:44:08.105795+0000","last_fullsized":"2024-03-27T08:44:08.105795+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.585915+0000","last_change":"2024-03-27T08:31:05.116401+0000","last_active":"2024-03-27T08:44:09.585915+0000","last_peered":"2024-03-27T08:44:09.585915+0000","last_clean":"2024-03-27T08:44:09.585915+0000","last_became_active":"2024-03-27T08:31:05.098437+0000","last_became_peered":"2024-03-27T08:31:05.098437+0000","last_unstale":"2024-03-27T08:44:09.585915+0000","last_undegraded":"2024-03-27T08:44:09.585915+0000","last_fullsized":"2024-03-27T08:44:09.585915+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.105080+0000","last_change":"2024-03-27T08:31:05.141787+0000","last_active":"2024-03-27T08:44:08.105080+0000","last_peered":"2024-03-27T08:44:08.105080+0000","last_clean":"2024-03-27T08:44:08.105080+0000","last_became_active":"2024-03-27T08:31:05.120236+0000","last_became_peered":"2024-03-27T08:31:05.120236+0000","last_unstale":"2024-03-27T08:44:08.105080+0000","last_undegraded":"2024-03-27T08:44:08.105080+0000","last_fullsized":"2024-03-27T08:44:08.105080+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.566589+0000","last_change":"2024-03-27T08:31:05.209271+0000","last_active":"2024-03-27T08:44:09.566589+0000","last_peered":"2024-03-27T08:44:09.566589+0000","last_clean":"2024-03-27T08:44:09.566589+0000","last_became_active":"2024-03-27T08:31:05.203943+0000","last_became_peered":"2024-03-27T08:31:05.203943+0000","last_unstale":"2024-03-27T08:44:09.566589+0000","last_undegraded":"2024-03-27T08:44:09.566589+0000","last_fullsized":"2024-03-27T08:44:09.566589+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.545449+0000","last_change":"2024-03-27T08:31:05.095455+0000","last_active":"2024-03-27T08:44:09.545449+0000","last_peered":"2024-03-27T08:44:09.545449+0000","last_clean":"2024-03-27T08:44:09.545449+0000","last_became_active":"2024-03-27T08:31:05.072116+0000","last_became_peered":"2024-03-27T08:31:05.072116+0000","last_unstale":"2024-03-27T08:44:09.545449+0000","last_undegraded":"2024-03-27T08:44:09.545449+0000","last_fullsized":"2024-03-27T08:44:09.545449+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.585098+0000","last_change":"2024-03-27T08:31:05.158859+0000","last_active":"2024-03-27T08:44:09.585098+0000","last_peered":"2024-03-27T08:44:09.585098+0000","last_clean":"2024-03-27T08:44:09.585098+0000","last_became_active":"2024-03-27T08:31:05.151468+0000","last_became_peered":"2024-03-27T08:31:05.151468+0000","last_unstale":"2024-03-27T08:44:09.585098+0000","last_undegraded":"2024-03-27T08:44:09.585098+0000","last_fullsized":"2024-03-27T08:44:09.585098+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":1294,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.433217+0000","last_change":"2024-03-27T08:31:05.069384+0000","last_active":"2024-03-27T08:44:09.433217+0000","last_peered":"2024-03-27T08:44:09.433217+0000","last_clean":"2024-03-27T08:44:09.433217+0000","last_became_active":"2024-03-27T08:31:05.058450+0000","last_became_peered":"2024-03-27T08:31:05.058450+0000","last_unstale":"2024-03-27T08:44:09.433217+0000","last_undegraded":"2024-03-27T08:44:09.433217+0000","last_fullsized":"2024-03-27T08:44:09.433217+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.346226+0000","last_change":"2024-03-27T08:31:05.085448+0000","last_active":"2024-03-27T08:44:09.346226+0000","last_peered":"2024-03-27T08:44:09.346226+0000","last_clean":"2024-03-27T08:44:09.346226+0000","last_became_active":"2024-03-27T08:31:05.070139+0000","last_became_peered":"2024-03-27T08:31:05.070139+0000","last_unstale":"2024-03-27T08:44:09.346226+0000","last_undegraded":"2024-03-27T08:44:09.346226+0000","last_fullsized":"2024-03-27T08:44:09.346226+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.500672+0000","last_change":"2024-03-27T08:26:13.349679+0000","last_active":"2024-03-27T08:44:09.500672+0000","last_peered":"2024-03-27T08:44:09.500672+0000","last_clean":"2024-03-27T08:44:09.500672+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:44:09.500672+0000","last_undegraded":"2024-03-27T08:44:09.500672+0000","last_fullsized":"2024-03-27T08:44:09.500672+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1857,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.397617+0000","last_change":"2024-03-27T08:26:13.342923+0000","last_active":"2024-03-27T08:44:09.397617+0000","last_peered":"2024-03-27T08:44:09.397617+0000","last_clean":"2024-03-27T08:44:09.397617+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:44:09.397617+0000","last_undegraded":"2024-03-27T08:44:09.397617+0000","last_fullsized":"2024-03-27T08:44:09.397617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1861,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.152693+0000","last_change":"2024-03-27T08:26:13.346210+0000","last_active":"2024-03-27T08:44:08.152693+0000","last_peered":"2024-03-27T08:44:08.152693+0000","last_clean":"2024-03-27T08:44:08.152693+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:44:08.152693+0000","last_undegraded":"2024-03-27T08:44:08.152693+0000","last_fullsized":"2024-03-27T08:44:08.152693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1857,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.330575+0000","last_change":"2024-03-27T08:26:13.333247+0000","last_active":"2024-03-27T08:44:09.330575+0000","last_peered":"2024-03-27T08:44:09.330575+0000","last_clean":"2024-03-27T08:44:09.330575+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:44:09.330575+0000","last_undegraded":"2024-03-27T08:44:09.330575+0000","last_fullsized":"2024-03-27T08:44:09.330575+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1845,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.660928+0000","last_change":"2024-03-27T08:26:13.347337+0000","last_active":"2024-03-27T08:44:09.660928+0000","last_peered":"2024-03-27T08:44:09.660928+0000","last_clean":"2024-03-27T08:44:09.660928+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:44:09.660928+0000","last_undegraded":"2024-03-27T08:44:09.660928+0000","last_fullsized":"2024-03-27T08:44:09.660928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.430296+0000","last_change":"2024-03-27T08:26:13.336420+0000","last_active":"2024-03-27T08:44:09.430296+0000","last_peered":"2024-03-27T08:44:09.430296+0000","last_clean":"2024-03-27T08:44:09.430296+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:44:09.430296+0000","last_undegraded":"2024-03-27T08:44:09.430296+0000","last_fullsized":"2024-03-27T08:44:09.430296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1845,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.650321+0000","last_change":"2024-03-27T08:26:13.341429+0000","last_active":"2024-03-27T08:44:09.650321+0000","last_peered":"2024-03-27T08:44:09.650321+0000","last_clean":"2024-03-27T08:44:09.650321+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:44:09.650321+0000","last_undegraded":"2024-03-27T08:44:09.650321+0000","last_fullsized":"2024-03-27T08:44:09.650321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.503645+0000","last_change":"2024-03-27T08:26:13.347502+0000","last_active":"2024-03-27T08:44:09.503645+0000","last_peered":"2024-03-27T08:44:09.503645+0000","last_clean":"2024-03-27T08:44:09.503645+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:44:09.503645+0000","last_undegraded":"2024-03-27T08:44:09.503645+0000","last_fullsized":"2024-03-27T08:44:09.503645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1994,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.086498+0000","last_change":"2024-03-27T08:26:11.311391+0000","last_active":"2024-03-27T08:44:08.086498+0000","last_peered":"2024-03-27T08:44:08.086498+0000","last_clean":"2024-03-27T08:44:08.086498+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:44:08.086498+0000","last_undegraded":"2024-03-27T08:44:08.086498+0000","last_fullsized":"2024-03-27T08:44:08.086498+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79076,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778524,"statfs":{"total":107374182400,"available":107293208576,"internally_reserved":0,"allocated":3633152,"data_stored":1464155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66592,"kb_used_data":3544,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791008,"statfs":{"total":107374182400,"available":107305992192,"internally_reserved":0,"allocated":3629056,"data_stored":1459999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83808,"kb_used_data":4120,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773792,"statfs":{"total":107374182400,"available":107288363008,"internally_reserved":0,"allocated":4218880,"data_stored":2065809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":67104,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104790496,"statfs":{"total":107374182400,"available":107305467904,"internally_reserved":0,"allocated":4153344,"data_stored":2018170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:44:41.140 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:44:41.249 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.250+0000 7fdfbbdd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:41.249 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.255+0000 7fdfbbdd7640 1 Processor -- start 2024-03-27T08:44:41.250 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.255+0000 7fdfbbdd7640 1 -- start start 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfbbdd7640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdfb412da20 0x7fdfb409c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfbbdd7640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb412e2b0 con 0x7fdfb409ca40 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfbbdd7640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb406e4d0 con 0x7fdfb412da20 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfb9b73640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fdfb412da20 0x7fdfb409c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfb9372640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fdfb409ca40 0x7fdfb41309a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43184/0 (socket says 172.21.15.104:43184) 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfb9372640 1 -- 172.21.15.104:0/1841566852 learned_addr learned my addr 172.21.15.104:0/1841566852 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.256+0000 7fdfb9b73640 1 --2- 172.21.15.104:0/1841566852 >> v2:172.21.15.104:3300/0 conn(0x7fdfb412da20 0x7fdfb409c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42284/0 (socket says 172.21.15.104:42284) 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.257+0000 7fdfb9b73640 1 -- 172.21.15.104:0/1841566852 >> v1:172.21.15.104:6789/0 conn(0x7fdfb409ca40 legacy=0x7fdfb41309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.257+0000 7fdfb9b73640 1 -- 172.21.15.104:0/1841566852 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb412dfa0 con 0x7fdfb412da20 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.257+0000 7fdfb9b73640 1 --2- 172.21.15.104:0/1841566852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb412da20 0x7fdfb409c500 crc :-1 s=READY pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1382bf3e8ba8993b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.257+0000 7fdfb8b71640 1 -- 172.21.15.104:0/1841566852 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdfac00e040 con 0x7fdfb412da20 2024-03-27T08:44:41.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.257+0000 7fdfb8b71640 1 -- 172.21.15.104:0/1841566852 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdfac007670 con 0x7fdfb412da20 2024-03-27T08:44:41.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.257+0000 7fdfb8b71640 1 -- 172.21.15.104:0/1841566852 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdfac007910 con 0x7fdfb412da20 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/1841566852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb412da20 msgr2=0x7fdfb409c500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 --2- 172.21.15.104:0/1841566852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb412da20 0x7fdfb409c500 crc :-1 s=READY pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/1841566852 shutdown_connections 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 --2- 172.21.15.104:0/1841566852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb412da20 0x7fdfb409c500 unknown :-1 s=CLOSED pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/1841566852 >> 172.21.15.104:0/1841566852 conn(0x7fdfb409d1d0 msgr2=0x7fdfb409d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/1841566852 shutdown_connections 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/1841566852 wait complete. 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.258+0000 7fdfbbdd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:41.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.259+0000 7fdfbbdd7640 1 Processor -- start 2024-03-27T08:44:41.255 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.259+0000 7fdfbbdd7640 1 -- start start 2024-03-27T08:44:41.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.260+0000 7fdfbbdd7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 0x7fdfb4119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.260+0000 7fdfbbdd7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb406a9e0 con 0x7fdfb409ca40 2024-03-27T08:44:41.257 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.260+0000 7fdfb9b73640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 0x7fdfb4119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.260+0000 7fdfb9b73640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 0x7fdfb4119fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42290/0 (socket says 172.21.15.104:42290) 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.260+0000 7fdfb9b73640 1 -- 172.21.15.104:0/81480748 learned_addr learned my addr 172.21.15.104:0/81480748 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfb9b73640 1 -- 172.21.15.104:0/81480748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb4119980 con 0x7fdfb409ca40 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfb9b73640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 0x7fdfb4119fd0 crc :-1 s=READY pgs=12985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdfac00e040 con 0x7fdfb409ca40 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdfac012a80 con 0x7fdfb409ca40 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb411d540 con 0x7fdfb409ca40 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdfac012d20 con 0x7fdfb409ca40 2024-03-27T08:44:41.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.261+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb411c700 con 0x7fdfb409ca40 2024-03-27T08:44:41.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.262+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fdfac0153f0 con 0x7fdfb409ca40 2024-03-27T08:44:41.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.262+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb411c8f0 con 0x7fdfb409ca40 2024-03-27T08:44:41.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.262+0000 7fdfa67fc640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdf980387a0 0x7fdf9803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.262+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 397..712) ==== 3403+0+0 (crc 0 0 0) 0x7fdfac049bd0 con 0x7fdfb409ca40 2024-03-27T08:44:41.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.262+0000 7fdfb9372640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdf980387a0 0x7fdf9803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.259 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.263+0000 7fdfb9372640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdf980387a0 0x7fdf9803acb0 crc :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.262 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.267+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fdfac00fc80 con 0x7fdfb409ca40 2024-03-27T08:44:41.436 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.442+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfb409ce50 con 0x7fdf980387a0 2024-03-27T08:44:41.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.445+0000 7fdfa67fc640 1 -- 172.21.15.104:0/81480748 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7fdfa8005910 con 0x7fdf980387a0 2024-03-27T08:44:41.441 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:44:41.441 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.447+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdf980387a0 msgr2=0x7fdf9803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdf980387a0 0x7fdf9803acb0 crc :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 msgr2=0x7fdfb4119fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 0x7fdfb4119fd0 crc :-1 s=READY pgs=12985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 shutdown_connections 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fdf980387a0 0x7fdf9803acb0 unknown :-1 s=CLOSED pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 --2- 172.21.15.104:0/81480748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb409ca40 0x7fdfb4119fd0 unknown :-1 s=CLOSED pgs=12985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 >> 172.21.15.104:0/81480748 conn(0x7fdfb409d1d0 msgr2=0x7fdfb409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 shutdown_connections 2024-03-27T08:44:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.448+0000 7fdfbbdd7640 1 -- 172.21.15.104:0/81480748 wait complete. 2024-03-27T08:44:41.454 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":1262,"stamp":"2024-03-27T08:44:40.476347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":296580,"kb_used_data":15268,"kb_used_omap":0,"kb_used_meta":281280,"kb_avail":419133820,"statfs":{"total":429496729600,"available":429193031680,"internally_reserved":0,"allocated":15634432,"data_stored":7008133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":288030720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002716"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.403955+0000","last_change":"2024-03-27T08:31:05.063650+0000","last_active":"2024-03-27T08:44:09.403955+0000","last_peered":"2024-03-27T08:44:09.403955+0000","last_clean":"2024-03-27T08:44:09.403955+0000","last_became_active":"2024-03-27T08:31:05.053480+0000","last_became_peered":"2024-03-27T08:31:05.053480+0000","last_unstale":"2024-03-27T08:44:09.403955+0000","last_undegraded":"2024-03-27T08:44:09.403955+0000","last_fullsized":"2024-03-27T08:44:09.403955+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.535366+0000","last_change":"2024-03-27T08:31:05.114356+0000","last_active":"2024-03-27T08:44:09.535366+0000","last_peered":"2024-03-27T08:44:09.535366+0000","last_clean":"2024-03-27T08:44:09.535366+0000","last_became_active":"2024-03-27T08:31:05.102560+0000","last_became_peered":"2024-03-27T08:31:05.102560+0000","last_unstale":"2024-03-27T08:44:09.535366+0000","last_undegraded":"2024-03-27T08:44:09.535366+0000","last_fullsized":"2024-03-27T08:44:09.535366+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.352281+0000","last_change":"2024-03-27T08:31:05.110354+0000","last_active":"2024-03-27T08:44:09.352281+0000","last_peered":"2024-03-27T08:44:09.352281+0000","last_clean":"2024-03-27T08:44:09.352281+0000","last_became_active":"2024-03-27T08:31:05.101799+0000","last_became_peered":"2024-03-27T08:31:05.101799+0000","last_unstale":"2024-03-27T08:44:09.352281+0000","last_undegraded":"2024-03-27T08:44:09.352281+0000","last_fullsized":"2024-03-27T08:44:09.352281+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.085002+0000","last_change":"2024-03-27T08:31:05.128758+0000","last_active":"2024-03-27T08:44:08.085002+0000","last_peered":"2024-03-27T08:44:08.085002+0000","last_clean":"2024-03-27T08:44:08.085002+0000","last_became_active":"2024-03-27T08:31:05.111183+0000","last_became_peered":"2024-03-27T08:31:05.111183+0000","last_unstale":"2024-03-27T08:44:08.085002+0000","last_undegraded":"2024-03-27T08:44:08.085002+0000","last_fullsized":"2024-03-27T08:44:08.085002+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.402434+0000","last_change":"2024-03-27T08:31:05.145624+0000","last_active":"2024-03-27T08:44:09.402434+0000","last_peered":"2024-03-27T08:44:09.402434+0000","last_clean":"2024-03-27T08:44:09.402434+0000","last_became_active":"2024-03-27T08:31:05.138042+0000","last_became_peered":"2024-03-27T08:31:05.138042+0000","last_unstale":"2024-03-27T08:44:09.402434+0000","last_undegraded":"2024-03-27T08:44:09.402434+0000","last_fullsized":"2024-03-27T08:44:09.402434+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.583640+0000","last_change":"2024-03-27T08:31:05.084677+0000","last_active":"2024-03-27T08:44:09.583640+0000","last_peered":"2024-03-27T08:44:09.583640+0000","last_clean":"2024-03-27T08:44:09.583640+0000","last_became_active":"2024-03-27T08:31:05.068818+0000","last_became_peered":"2024-03-27T08:31:05.068818+0000","last_unstale":"2024-03-27T08:44:09.583640+0000","last_undegraded":"2024-03-27T08:44:09.583640+0000","last_fullsized":"2024-03-27T08:44:09.583640+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.503546+0000","last_change":"2024-03-27T08:31:05.087048+0000","last_active":"2024-03-27T08:44:09.503546+0000","last_peered":"2024-03-27T08:44:09.503546+0000","last_clean":"2024-03-27T08:44:09.503546+0000","last_became_active":"2024-03-27T08:31:05.064801+0000","last_became_peered":"2024-03-27T08:31:05.064801+0000","last_unstale":"2024-03-27T08:44:09.503546+0000","last_undegraded":"2024-03-27T08:44:09.503546+0000","last_fullsized":"2024-03-27T08:44:09.503546+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.516989+0000","last_change":"2024-03-27T08:31:05.134669+0000","last_active":"2024-03-27T08:44:09.516989+0000","last_peered":"2024-03-27T08:44:09.516989+0000","last_clean":"2024-03-27T08:44:09.516989+0000","last_became_active":"2024-03-27T08:31:05.128790+0000","last_became_peered":"2024-03-27T08:31:05.128790+0000","last_unstale":"2024-03-27T08:44:09.516989+0000","last_undegraded":"2024-03-27T08:44:09.516989+0000","last_fullsized":"2024-03-27T08:44:09.516989+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.111329+0000","last_change":"2024-03-27T08:31:05.144335+0000","last_active":"2024-03-27T08:44:08.111329+0000","last_peered":"2024-03-27T08:44:08.111329+0000","last_clean":"2024-03-27T08:44:08.111329+0000","last_became_active":"2024-03-27T08:31:05.125515+0000","last_became_peered":"2024-03-27T08:31:05.125515+0000","last_unstale":"2024-03-27T08:44:08.111329+0000","last_undegraded":"2024-03-27T08:44:08.111329+0000","last_fullsized":"2024-03-27T08:44:08.111329+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.368034+0000","last_change":"2024-03-27T08:31:05.083033+0000","last_active":"2024-03-27T08:44:09.368034+0000","last_peered":"2024-03-27T08:44:09.368034+0000","last_clean":"2024-03-27T08:44:09.368034+0000","last_became_active":"2024-03-27T08:31:05.074459+0000","last_became_peered":"2024-03-27T08:31:05.074459+0000","last_unstale":"2024-03-27T08:44:09.368034+0000","last_undegraded":"2024-03-27T08:44:09.368034+0000","last_fullsized":"2024-03-27T08:44:09.368034+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.112194+0000","last_change":"2024-03-27T08:31:05.114956+0000","last_active":"2024-03-27T08:44:08.112194+0000","last_peered":"2024-03-27T08:44:08.112194+0000","last_clean":"2024-03-27T08:44:08.112194+0000","last_became_active":"2024-03-27T08:31:05.099066+0000","last_became_peered":"2024-03-27T08:31:05.099066+0000","last_unstale":"2024-03-27T08:44:08.112194+0000","last_undegraded":"2024-03-27T08:44:08.112194+0000","last_fullsized":"2024-03-27T08:44:08.112194+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.577379+0000","last_change":"2024-03-27T08:31:05.179980+0000","last_active":"2024-03-27T08:44:09.577379+0000","last_peered":"2024-03-27T08:44:09.577379+0000","last_clean":"2024-03-27T08:44:09.577379+0000","last_became_active":"2024-03-27T08:31:05.174215+0000","last_became_peered":"2024-03-27T08:31:05.174215+0000","last_unstale":"2024-03-27T08:44:09.577379+0000","last_undegraded":"2024-03-27T08:44:09.577379+0000","last_fullsized":"2024-03-27T08:44:09.577379+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.106888+0000","last_change":"2024-03-27T08:31:05.108652+0000","last_active":"2024-03-27T08:44:08.106888+0000","last_peered":"2024-03-27T08:44:08.106888+0000","last_clean":"2024-03-27T08:44:08.106888+0000","last_became_active":"2024-03-27T08:31:05.095788+0000","last_became_peered":"2024-03-27T08:31:05.095788+0000","last_unstale":"2024-03-27T08:44:08.106888+0000","last_undegraded":"2024-03-27T08:44:08.106888+0000","last_fullsized":"2024-03-27T08:44:08.106888+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.081647+0000","last_change":"2024-03-27T08:31:05.131740+0000","last_active":"2024-03-27T08:44:08.081647+0000","last_peered":"2024-03-27T08:44:08.081647+0000","last_clean":"2024-03-27T08:44:08.081647+0000","last_became_active":"2024-03-27T08:31:05.103731+0000","last_became_peered":"2024-03-27T08:31:05.103731+0000","last_unstale":"2024-03-27T08:44:08.081647+0000","last_undegraded":"2024-03-27T08:44:08.081647+0000","last_fullsized":"2024-03-27T08:44:08.081647+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.543849+0000","last_change":"2024-03-27T08:31:05.119311+0000","last_active":"2024-03-27T08:44:09.543849+0000","last_peered":"2024-03-27T08:44:09.543849+0000","last_clean":"2024-03-27T08:44:09.543849+0000","last_became_active":"2024-03-27T08:31:05.090562+0000","last_became_peered":"2024-03-27T08:31:05.090562+0000","last_unstale":"2024-03-27T08:44:09.543849+0000","last_undegraded":"2024-03-27T08:44:09.543849+0000","last_fullsized":"2024-03-27T08:44:09.543849+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.573731+0000","last_change":"2024-03-27T08:31:05.106623+0000","last_active":"2024-03-27T08:44:09.573731+0000","last_peered":"2024-03-27T08:44:09.573731+0000","last_clean":"2024-03-27T08:44:09.573731+0000","last_became_active":"2024-03-27T08:31:05.076510+0000","last_became_peered":"2024-03-27T08:31:05.076510+0000","last_unstale":"2024-03-27T08:44:09.573731+0000","last_undegraded":"2024-03-27T08:44:09.573731+0000","last_fullsized":"2024-03-27T08:44:09.573731+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.582170+0000","last_change":"2024-03-27T08:31:05.110196+0000","last_active":"2024-03-27T08:44:09.582170+0000","last_peered":"2024-03-27T08:44:09.582170+0000","last_clean":"2024-03-27T08:44:09.582170+0000","last_became_active":"2024-03-27T08:31:05.081587+0000","last_became_peered":"2024-03-27T08:31:05.081587+0000","last_unstale":"2024-03-27T08:44:09.582170+0000","last_undegraded":"2024-03-27T08:44:09.582170+0000","last_fullsized":"2024-03-27T08:44:09.582170+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.114526+0000","last_change":"2024-03-27T08:31:05.153706+0000","last_active":"2024-03-27T08:44:08.114526+0000","last_peered":"2024-03-27T08:44:08.114526+0000","last_clean":"2024-03-27T08:44:08.114526+0000","last_became_active":"2024-03-27T08:31:05.137462+0000","last_became_peered":"2024-03-27T08:31:05.137462+0000","last_unstale":"2024-03-27T08:44:08.114526+0000","last_undegraded":"2024-03-27T08:44:08.114526+0000","last_fullsized":"2024-03-27T08:44:08.114526+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.357491+0000","last_change":"2024-03-27T08:31:05.091071+0000","last_active":"2024-03-27T08:44:09.357491+0000","last_peered":"2024-03-27T08:44:09.357491+0000","last_clean":"2024-03-27T08:44:09.357491+0000","last_became_active":"2024-03-27T08:31:05.077928+0000","last_became_peered":"2024-03-27T08:31:05.077928+0000","last_unstale":"2024-03-27T08:44:09.357491+0000","last_undegraded":"2024-03-27T08:44:09.357491+0000","last_fullsized":"2024-03-27T08:44:09.357491+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.406084+0000","last_change":"2024-03-27T08:31:05.096352+0000","last_active":"2024-03-27T08:44:09.406084+0000","last_peered":"2024-03-27T08:44:09.406084+0000","last_clean":"2024-03-27T08:44:09.406084+0000","last_became_active":"2024-03-27T08:31:05.090906+0000","last_became_peered":"2024-03-27T08:31:05.090906+0000","last_unstale":"2024-03-27T08:44:09.406084+0000","last_undegraded":"2024-03-27T08:44:09.406084+0000","last_fullsized":"2024-03-27T08:44:09.406084+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.412305+0000","last_change":"2024-03-27T08:31:05.198445+0000","last_active":"2024-03-27T08:44:09.412305+0000","last_peered":"2024-03-27T08:44:09.412305+0000","last_clean":"2024-03-27T08:44:09.412305+0000","last_became_active":"2024-03-27T08:31:05.192295+0000","last_became_peered":"2024-03-27T08:31:05.192295+0000","last_unstale":"2024-03-27T08:44:09.412305+0000","last_undegraded":"2024-03-27T08:44:09.412305+0000","last_fullsized":"2024-03-27T08:44:09.412305+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.448126+0000","last_change":"2024-03-27T08:31:05.085798+0000","last_active":"2024-03-27T08:44:09.448126+0000","last_peered":"2024-03-27T08:44:09.448126+0000","last_clean":"2024-03-27T08:44:09.448126+0000","last_became_active":"2024-03-27T08:31:05.077394+0000","last_became_peered":"2024-03-27T08:31:05.077394+0000","last_unstale":"2024-03-27T08:44:09.448126+0000","last_undegraded":"2024-03-27T08:44:09.448126+0000","last_fullsized":"2024-03-27T08:44:09.448126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.347767+0000","last_change":"2024-03-27T08:31:05.120108+0000","last_active":"2024-03-27T08:44:09.347767+0000","last_peered":"2024-03-27T08:44:09.347767+0000","last_clean":"2024-03-27T08:44:09.347767+0000","last_became_active":"2024-03-27T08:31:05.114332+0000","last_became_peered":"2024-03-27T08:31:05.114332+0000","last_unstale":"2024-03-27T08:44:09.347767+0000","last_undegraded":"2024-03-27T08:44:09.347767+0000","last_fullsized":"2024-03-27T08:44:09.347767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.074802+0000","last_change":"2024-03-27T08:31:05.181415+0000","last_active":"2024-03-27T08:44:08.074802+0000","last_peered":"2024-03-27T08:44:08.074802+0000","last_clean":"2024-03-27T08:44:08.074802+0000","last_became_active":"2024-03-27T08:31:05.175030+0000","last_became_peered":"2024-03-27T08:31:05.175030+0000","last_unstale":"2024-03-27T08:44:08.074802+0000","last_undegraded":"2024-03-27T08:44:08.074802+0000","last_fullsized":"2024-03-27T08:44:08.074802+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.105795+0000","last_change":"2024-03-27T08:31:05.169539+0000","last_active":"2024-03-27T08:44:08.105795+0000","last_peered":"2024-03-27T08:44:08.105795+0000","last_clean":"2024-03-27T08:44:08.105795+0000","last_became_active":"2024-03-27T08:31:05.164396+0000","last_became_peered":"2024-03-27T08:31:05.164396+0000","last_unstale":"2024-03-27T08:44:08.105795+0000","last_undegraded":"2024-03-27T08:44:08.105795+0000","last_fullsized":"2024-03-27T08:44:08.105795+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.585915+0000","last_change":"2024-03-27T08:31:05.116401+0000","last_active":"2024-03-27T08:44:09.585915+0000","last_peered":"2024-03-27T08:44:09.585915+0000","last_clean":"2024-03-27T08:44:09.585915+0000","last_became_active":"2024-03-27T08:31:05.098437+0000","last_became_peered":"2024-03-27T08:31:05.098437+0000","last_unstale":"2024-03-27T08:44:09.585915+0000","last_undegraded":"2024-03-27T08:44:09.585915+0000","last_fullsized":"2024-03-27T08:44:09.585915+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.105080+0000","last_change":"2024-03-27T08:31:05.141787+0000","last_active":"2024-03-27T08:44:08.105080+0000","last_peered":"2024-03-27T08:44:08.105080+0000","last_clean":"2024-03-27T08:44:08.105080+0000","last_became_active":"2024-03-27T08:31:05.120236+0000","last_became_peered":"2024-03-27T08:31:05.120236+0000","last_unstale":"2024-03-27T08:44:08.105080+0000","last_undegraded":"2024-03-27T08:44:08.105080+0000","last_fullsized":"2024-03-27T08:44:08.105080+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.566589+0000","last_change":"2024-03-27T08:31:05.209271+0000","last_active":"2024-03-27T08:44:09.566589+0000","last_peered":"2024-03-27T08:44:09.566589+0000","last_clean":"2024-03-27T08:44:09.566589+0000","last_became_active":"2024-03-27T08:31:05.203943+0000","last_became_peered":"2024-03-27T08:31:05.203943+0000","last_unstale":"2024-03-27T08:44:09.566589+0000","last_undegraded":"2024-03-27T08:44:09.566589+0000","last_fullsized":"2024-03-27T08:44:09.566589+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.545449+0000","last_change":"2024-03-27T08:31:05.095455+0000","last_active":"2024-03-27T08:44:09.545449+0000","last_peered":"2024-03-27T08:44:09.545449+0000","last_clean":"2024-03-27T08:44:09.545449+0000","last_became_active":"2024-03-27T08:31:05.072116+0000","last_became_peered":"2024-03-27T08:31:05.072116+0000","last_unstale":"2024-03-27T08:44:09.545449+0000","last_undegraded":"2024-03-27T08:44:09.545449+0000","last_fullsized":"2024-03-27T08:44:09.545449+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.585098+0000","last_change":"2024-03-27T08:31:05.158859+0000","last_active":"2024-03-27T08:44:09.585098+0000","last_peered":"2024-03-27T08:44:09.585098+0000","last_clean":"2024-03-27T08:44:09.585098+0000","last_became_active":"2024-03-27T08:31:05.151468+0000","last_became_peered":"2024-03-27T08:31:05.151468+0000","last_unstale":"2024-03-27T08:44:09.585098+0000","last_undegraded":"2024-03-27T08:44:09.585098+0000","last_fullsized":"2024-03-27T08:44:09.585098+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":1294,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.433217+0000","last_change":"2024-03-27T08:31:05.069384+0000","last_active":"2024-03-27T08:44:09.433217+0000","last_peered":"2024-03-27T08:44:09.433217+0000","last_clean":"2024-03-27T08:44:09.433217+0000","last_became_active":"2024-03-27T08:31:05.058450+0000","last_became_peered":"2024-03-27T08:31:05.058450+0000","last_unstale":"2024-03-27T08:44:09.433217+0000","last_undegraded":"2024-03-27T08:44:09.433217+0000","last_fullsized":"2024-03-27T08:44:09.433217+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.346226+0000","last_change":"2024-03-27T08:31:05.085448+0000","last_active":"2024-03-27T08:44:09.346226+0000","last_peered":"2024-03-27T08:44:09.346226+0000","last_clean":"2024-03-27T08:44:09.346226+0000","last_became_active":"2024-03-27T08:31:05.070139+0000","last_became_peered":"2024-03-27T08:31:05.070139+0000","last_unstale":"2024-03-27T08:44:09.346226+0000","last_undegraded":"2024-03-27T08:44:09.346226+0000","last_fullsized":"2024-03-27T08:44:09.346226+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.500672+0000","last_change":"2024-03-27T08:26:13.349679+0000","last_active":"2024-03-27T08:44:09.500672+0000","last_peered":"2024-03-27T08:44:09.500672+0000","last_clean":"2024-03-27T08:44:09.500672+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:44:09.500672+0000","last_undegraded":"2024-03-27T08:44:09.500672+0000","last_fullsized":"2024-03-27T08:44:09.500672+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1857,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.397617+0000","last_change":"2024-03-27T08:26:13.342923+0000","last_active":"2024-03-27T08:44:09.397617+0000","last_peered":"2024-03-27T08:44:09.397617+0000","last_clean":"2024-03-27T08:44:09.397617+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:44:09.397617+0000","last_undegraded":"2024-03-27T08:44:09.397617+0000","last_fullsized":"2024-03-27T08:44:09.397617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1861,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.152693+0000","last_change":"2024-03-27T08:26:13.346210+0000","last_active":"2024-03-27T08:44:08.152693+0000","last_peered":"2024-03-27T08:44:08.152693+0000","last_clean":"2024-03-27T08:44:08.152693+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:44:08.152693+0000","last_undegraded":"2024-03-27T08:44:08.152693+0000","last_fullsized":"2024-03-27T08:44:08.152693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1857,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.330575+0000","last_change":"2024-03-27T08:26:13.333247+0000","last_active":"2024-03-27T08:44:09.330575+0000","last_peered":"2024-03-27T08:44:09.330575+0000","last_clean":"2024-03-27T08:44:09.330575+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:44:09.330575+0000","last_undegraded":"2024-03-27T08:44:09.330575+0000","last_fullsized":"2024-03-27T08:44:09.330575+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1845,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.660928+0000","last_change":"2024-03-27T08:26:13.347337+0000","last_active":"2024-03-27T08:44:09.660928+0000","last_peered":"2024-03-27T08:44:09.660928+0000","last_clean":"2024-03-27T08:44:09.660928+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:44:09.660928+0000","last_undegraded":"2024-03-27T08:44:09.660928+0000","last_fullsized":"2024-03-27T08:44:09.660928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.430296+0000","last_change":"2024-03-27T08:26:13.336420+0000","last_active":"2024-03-27T08:44:09.430296+0000","last_peered":"2024-03-27T08:44:09.430296+0000","last_clean":"2024-03-27T08:44:09.430296+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:44:09.430296+0000","last_undegraded":"2024-03-27T08:44:09.430296+0000","last_fullsized":"2024-03-27T08:44:09.430296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1845,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.650321+0000","last_change":"2024-03-27T08:26:13.341429+0000","last_active":"2024-03-27T08:44:09.650321+0000","last_peered":"2024-03-27T08:44:09.650321+0000","last_clean":"2024-03-27T08:44:09.650321+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:44:09.650321+0000","last_undegraded":"2024-03-27T08:44:09.650321+0000","last_fullsized":"2024-03-27T08:44:09.650321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.503645+0000","last_change":"2024-03-27T08:26:13.347502+0000","last_active":"2024-03-27T08:44:09.503645+0000","last_peered":"2024-03-27T08:44:09.503645+0000","last_clean":"2024-03-27T08:44:09.503645+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:44:09.503645+0000","last_undegraded":"2024-03-27T08:44:09.503645+0000","last_fullsized":"2024-03-27T08:44:09.503645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1994,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.086498+0000","last_change":"2024-03-27T08:26:11.311391+0000","last_active":"2024-03-27T08:44:08.086498+0000","last_peered":"2024-03-27T08:44:08.086498+0000","last_clean":"2024-03-27T08:44:08.086498+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:44:08.086498+0000","last_undegraded":"2024-03-27T08:44:08.086498+0000","last_fullsized":"2024-03-27T08:44:08.086498+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79076,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778524,"statfs":{"total":107374182400,"available":107293208576,"internally_reserved":0,"allocated":3633152,"data_stored":1464155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66592,"kb_used_data":3544,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791008,"statfs":{"total":107374182400,"available":107305992192,"internally_reserved":0,"allocated":3629056,"data_stored":1459999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83808,"kb_used_data":4120,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773792,"statfs":{"total":107374182400,"available":107288363008,"internally_reserved":0,"allocated":4218880,"data_stored":2065809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":67104,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104790496,"statfs":{"total":107374182400,"available":107305467904,"internally_reserved":0,"allocated":4153344,"data_stored":2018170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:44:41.455 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-27T08:44:41.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:44:41.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.574+0000 7f0cd7f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:41.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.580+0000 7f0cd7f9d640 1 Processor -- start 2024-03-27T08:44:41.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.580+0000 7f0cd7f9d640 1 -- start start 2024-03-27T08:44:41.576 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd7f9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0cd0131830 0x7f0cd0131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd7f9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd00735e0 con 0x7f0cd0134580 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd7f9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd00a0260 con 0x7f0cd0131830 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd5d39640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0cd0131830 0x7f0cd0131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd5538640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0cd0134580 0x7f0cd0134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43198/0 (socket says 172.21.15.104:43198) 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd5538640 1 -- 172.21.15.104:0/4081982167 learned_addr learned my addr 172.21.15.104:0/4081982167 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd5d39640 1 -- 172.21.15.104:0/4081982167 >> v1:172.21.15.104:6789/0 conn(0x7f0cd0134580 legacy=0x7f0cd0134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.582+0000 7f0cd5d39640 1 -- 172.21.15.104:0/4081982167 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd006eb80 con 0x7f0cd0131830 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.581+0000 7f0cd4d37640 1 -- 172.21.15.104:0/4081982167 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1723382726 0 0) 0x7f0cd00735e0 con 0x7f0cd0134580 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.582+0000 7f0cd5d39640 1 --2- 172.21.15.104:0/4081982167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0131c40 crc :-1 s=READY pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c5afb19e0ffa30 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.582+0000 7f0cd4d37640 1 -- 172.21.15.104:0/4081982167 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ccc00e040 con 0x7f0cd0131830 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.582+0000 7f0cd4d37640 1 -- 172.21.15.104:0/4081982167 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0ccc007670 con 0x7f0cd0131830 2024-03-27T08:44:41.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.582+0000 7f0cd4d37640 1 -- 172.21.15.104:0/4081982167 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ccc007910 con 0x7f0cd0131830 2024-03-27T08:44:41.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/4081982167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 msgr2=0x7f0cd0131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 --2- 172.21.15.104:0/4081982167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0131c40 crc :-1 s=READY pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/4081982167 shutdown_connections 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 --2- 172.21.15.104:0/4081982167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0131c40 unknown :-1 s=CLOSED pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/4081982167 >> 172.21.15.104:0/4081982167 conn(0x7f0cd00a4ae0 msgr2=0x7f0cd00a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/4081982167 shutdown_connections 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/4081982167 wait complete. 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.583+0000 7f0cd7f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.585+0000 7f0cd7f9d640 1 Processor -- start 2024-03-27T08:44:41.579 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.585+0000 7f0cd7f9d640 1 -- start start 2024-03-27T08:44:41.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd7f9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd7f9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0072be0 con 0x7f0cd0131830 2024-03-27T08:44:41.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd5d39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd5d39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42312/0 (socket says 172.21.15.104:42312) 2024-03-27T08:44:41.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd5d39640 1 -- 172.21.15.104:0/34772257 learned_addr learned my addr 172.21.15.104:0/34772257 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:41.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd5d39640 1 -- 172.21.15.104:0/34772257 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd011e5e0 con 0x7f0cd0131830 2024-03-27T08:44:41.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.586+0000 7f0cd5d39640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0070c50 crc :-1 s=READY pgs=12988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.587+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ccc00e040 con 0x7f0cd0131830 2024-03-27T08:44:41.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.587+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0ccc012b00 con 0x7f0cd0131830 2024-03-27T08:44:41.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.587+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd0072780 con 0x7f0cd0131830 2024-03-27T08:44:41.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.587+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ccc012da0 con 0x7f0cd0131830 2024-03-27T08:44:41.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.587+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd011e2d0 con 0x7f0cd0131830 2024-03-27T08:44:41.583 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.588+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f0ccc0153f0 con 0x7f0cd0131830 2024-03-27T08:44:41.583 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.588+0000 7f0cc67fc640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0cb4038750 0x7f0cb403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.583 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.588+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 397..712) ==== 3403+0+0 (crc 0 0 0) 0x7f0ccc049b90 con 0x7f0cd0131830 2024-03-27T08:44:41.583 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.588+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd011ef70 con 0x7f0cd0131830 2024-03-27T08:44:41.583 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.588+0000 7f0cd5538640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0cb4038750 0x7f0cb403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.589+0000 7f0cd5538640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0cb4038750 0x7f0cb403ac60 crc :-1 s=READY pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.588 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.594+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f0ccc00fe80 con 0x7f0cd0131830 2024-03-27T08:44:41.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.784+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cd009fe10 con 0x7f0cb4038750 2024-03-27T08:44:41.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.789+0000 7f0cc67fc640 1 -- 172.21.15.104:0/34772257 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7f0cd009fe10 con 0x7f0cb4038750 2024-03-27T08:44:41.784 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:44:41.785 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.791+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0cb4038750 msgr2=0x7f0cb403ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.791+0000 7f0cd7f9d640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0cb4038750 0x7f0cb403ac60 crc :-1 s=READY pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.791+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 msgr2=0x7f0cd0070c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.791+0000 7f0cd7f9d640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0070c50 crc :-1 s=READY pgs=12988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.792+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 shutdown_connections 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.792+0000 7f0cd7f9d640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f0cb4038750 0x7f0cb403ac60 unknown :-1 s=CLOSED pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.792+0000 7f0cd7f9d640 1 --2- 172.21.15.104:0/34772257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cd0131830 0x7f0cd0070c50 unknown :-1 s=CLOSED pgs=12988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.792+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 >> 172.21.15.104:0/34772257 conn(0x7f0cd00a4ae0 msgr2=0x7f0cd0132380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.792+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 shutdown_connections 2024-03-27T08:44:41.787 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.792+0000 7f0cd7f9d640 1 -- 172.21.15.104:0/34772257 wait complete. 2024-03-27T08:44:41.801 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":1262,"stamp":"2024-03-27T08:44:40.476347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":296580,"kb_used_data":15268,"kb_used_omap":0,"kb_used_meta":281280,"kb_avail":419133820,"statfs":{"total":429496729600,"available":429193031680,"internally_reserved":0,"allocated":15634432,"data_stored":7008133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":288030720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002716"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.403955+0000","last_change":"2024-03-27T08:31:05.063650+0000","last_active":"2024-03-27T08:44:09.403955+0000","last_peered":"2024-03-27T08:44:09.403955+0000","last_clean":"2024-03-27T08:44:09.403955+0000","last_became_active":"2024-03-27T08:31:05.053480+0000","last_became_peered":"2024-03-27T08:31:05.053480+0000","last_unstale":"2024-03-27T08:44:09.403955+0000","last_undegraded":"2024-03-27T08:44:09.403955+0000","last_fullsized":"2024-03-27T08:44:09.403955+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.535366+0000","last_change":"2024-03-27T08:31:05.114356+0000","last_active":"2024-03-27T08:44:09.535366+0000","last_peered":"2024-03-27T08:44:09.535366+0000","last_clean":"2024-03-27T08:44:09.535366+0000","last_became_active":"2024-03-27T08:31:05.102560+0000","last_became_peered":"2024-03-27T08:31:05.102560+0000","last_unstale":"2024-03-27T08:44:09.535366+0000","last_undegraded":"2024-03-27T08:44:09.535366+0000","last_fullsized":"2024-03-27T08:44:09.535366+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.352281+0000","last_change":"2024-03-27T08:31:05.110354+0000","last_active":"2024-03-27T08:44:09.352281+0000","last_peered":"2024-03-27T08:44:09.352281+0000","last_clean":"2024-03-27T08:44:09.352281+0000","last_became_active":"2024-03-27T08:31:05.101799+0000","last_became_peered":"2024-03-27T08:31:05.101799+0000","last_unstale":"2024-03-27T08:44:09.352281+0000","last_undegraded":"2024-03-27T08:44:09.352281+0000","last_fullsized":"2024-03-27T08:44:09.352281+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.085002+0000","last_change":"2024-03-27T08:31:05.128758+0000","last_active":"2024-03-27T08:44:08.085002+0000","last_peered":"2024-03-27T08:44:08.085002+0000","last_clean":"2024-03-27T08:44:08.085002+0000","last_became_active":"2024-03-27T08:31:05.111183+0000","last_became_peered":"2024-03-27T08:31:05.111183+0000","last_unstale":"2024-03-27T08:44:08.085002+0000","last_undegraded":"2024-03-27T08:44:08.085002+0000","last_fullsized":"2024-03-27T08:44:08.085002+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.402434+0000","last_change":"2024-03-27T08:31:05.145624+0000","last_active":"2024-03-27T08:44:09.402434+0000","last_peered":"2024-03-27T08:44:09.402434+0000","last_clean":"2024-03-27T08:44:09.402434+0000","last_became_active":"2024-03-27T08:31:05.138042+0000","last_became_peered":"2024-03-27T08:31:05.138042+0000","last_unstale":"2024-03-27T08:44:09.402434+0000","last_undegraded":"2024-03-27T08:44:09.402434+0000","last_fullsized":"2024-03-27T08:44:09.402434+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.583640+0000","last_change":"2024-03-27T08:31:05.084677+0000","last_active":"2024-03-27T08:44:09.583640+0000","last_peered":"2024-03-27T08:44:09.583640+0000","last_clean":"2024-03-27T08:44:09.583640+0000","last_became_active":"2024-03-27T08:31:05.068818+0000","last_became_peered":"2024-03-27T08:31:05.068818+0000","last_unstale":"2024-03-27T08:44:09.583640+0000","last_undegraded":"2024-03-27T08:44:09.583640+0000","last_fullsized":"2024-03-27T08:44:09.583640+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.503546+0000","last_change":"2024-03-27T08:31:05.087048+0000","last_active":"2024-03-27T08:44:09.503546+0000","last_peered":"2024-03-27T08:44:09.503546+0000","last_clean":"2024-03-27T08:44:09.503546+0000","last_became_active":"2024-03-27T08:31:05.064801+0000","last_became_peered":"2024-03-27T08:31:05.064801+0000","last_unstale":"2024-03-27T08:44:09.503546+0000","last_undegraded":"2024-03-27T08:44:09.503546+0000","last_fullsized":"2024-03-27T08:44:09.503546+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.516989+0000","last_change":"2024-03-27T08:31:05.134669+0000","last_active":"2024-03-27T08:44:09.516989+0000","last_peered":"2024-03-27T08:44:09.516989+0000","last_clean":"2024-03-27T08:44:09.516989+0000","last_became_active":"2024-03-27T08:31:05.128790+0000","last_became_peered":"2024-03-27T08:31:05.128790+0000","last_unstale":"2024-03-27T08:44:09.516989+0000","last_undegraded":"2024-03-27T08:44:09.516989+0000","last_fullsized":"2024-03-27T08:44:09.516989+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.111329+0000","last_change":"2024-03-27T08:31:05.144335+0000","last_active":"2024-03-27T08:44:08.111329+0000","last_peered":"2024-03-27T08:44:08.111329+0000","last_clean":"2024-03-27T08:44:08.111329+0000","last_became_active":"2024-03-27T08:31:05.125515+0000","last_became_peered":"2024-03-27T08:31:05.125515+0000","last_unstale":"2024-03-27T08:44:08.111329+0000","last_undegraded":"2024-03-27T08:44:08.111329+0000","last_fullsized":"2024-03-27T08:44:08.111329+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.368034+0000","last_change":"2024-03-27T08:31:05.083033+0000","last_active":"2024-03-27T08:44:09.368034+0000","last_peered":"2024-03-27T08:44:09.368034+0000","last_clean":"2024-03-27T08:44:09.368034+0000","last_became_active":"2024-03-27T08:31:05.074459+0000","last_became_peered":"2024-03-27T08:31:05.074459+0000","last_unstale":"2024-03-27T08:44:09.368034+0000","last_undegraded":"2024-03-27T08:44:09.368034+0000","last_fullsized":"2024-03-27T08:44:09.368034+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.112194+0000","last_change":"2024-03-27T08:31:05.114956+0000","last_active":"2024-03-27T08:44:08.112194+0000","last_peered":"2024-03-27T08:44:08.112194+0000","last_clean":"2024-03-27T08:44:08.112194+0000","last_became_active":"2024-03-27T08:31:05.099066+0000","last_became_peered":"2024-03-27T08:31:05.099066+0000","last_unstale":"2024-03-27T08:44:08.112194+0000","last_undegraded":"2024-03-27T08:44:08.112194+0000","last_fullsized":"2024-03-27T08:44:08.112194+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.577379+0000","last_change":"2024-03-27T08:31:05.179980+0000","last_active":"2024-03-27T08:44:09.577379+0000","last_peered":"2024-03-27T08:44:09.577379+0000","last_clean":"2024-03-27T08:44:09.577379+0000","last_became_active":"2024-03-27T08:31:05.174215+0000","last_became_peered":"2024-03-27T08:31:05.174215+0000","last_unstale":"2024-03-27T08:44:09.577379+0000","last_undegraded":"2024-03-27T08:44:09.577379+0000","last_fullsized":"2024-03-27T08:44:09.577379+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.106888+0000","last_change":"2024-03-27T08:31:05.108652+0000","last_active":"2024-03-27T08:44:08.106888+0000","last_peered":"2024-03-27T08:44:08.106888+0000","last_clean":"2024-03-27T08:44:08.106888+0000","last_became_active":"2024-03-27T08:31:05.095788+0000","last_became_peered":"2024-03-27T08:31:05.095788+0000","last_unstale":"2024-03-27T08:44:08.106888+0000","last_undegraded":"2024-03-27T08:44:08.106888+0000","last_fullsized":"2024-03-27T08:44:08.106888+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.081647+0000","last_change":"2024-03-27T08:31:05.131740+0000","last_active":"2024-03-27T08:44:08.081647+0000","last_peered":"2024-03-27T08:44:08.081647+0000","last_clean":"2024-03-27T08:44:08.081647+0000","last_became_active":"2024-03-27T08:31:05.103731+0000","last_became_peered":"2024-03-27T08:31:05.103731+0000","last_unstale":"2024-03-27T08:44:08.081647+0000","last_undegraded":"2024-03-27T08:44:08.081647+0000","last_fullsized":"2024-03-27T08:44:08.081647+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.543849+0000","last_change":"2024-03-27T08:31:05.119311+0000","last_active":"2024-03-27T08:44:09.543849+0000","last_peered":"2024-03-27T08:44:09.543849+0000","last_clean":"2024-03-27T08:44:09.543849+0000","last_became_active":"2024-03-27T08:31:05.090562+0000","last_became_peered":"2024-03-27T08:31:05.090562+0000","last_unstale":"2024-03-27T08:44:09.543849+0000","last_undegraded":"2024-03-27T08:44:09.543849+0000","last_fullsized":"2024-03-27T08:44:09.543849+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.573731+0000","last_change":"2024-03-27T08:31:05.106623+0000","last_active":"2024-03-27T08:44:09.573731+0000","last_peered":"2024-03-27T08:44:09.573731+0000","last_clean":"2024-03-27T08:44:09.573731+0000","last_became_active":"2024-03-27T08:31:05.076510+0000","last_became_peered":"2024-03-27T08:31:05.076510+0000","last_unstale":"2024-03-27T08:44:09.573731+0000","last_undegraded":"2024-03-27T08:44:09.573731+0000","last_fullsized":"2024-03-27T08:44:09.573731+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.582170+0000","last_change":"2024-03-27T08:31:05.110196+0000","last_active":"2024-03-27T08:44:09.582170+0000","last_peered":"2024-03-27T08:44:09.582170+0000","last_clean":"2024-03-27T08:44:09.582170+0000","last_became_active":"2024-03-27T08:31:05.081587+0000","last_became_peered":"2024-03-27T08:31:05.081587+0000","last_unstale":"2024-03-27T08:44:09.582170+0000","last_undegraded":"2024-03-27T08:44:09.582170+0000","last_fullsized":"2024-03-27T08:44:09.582170+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.114526+0000","last_change":"2024-03-27T08:31:05.153706+0000","last_active":"2024-03-27T08:44:08.114526+0000","last_peered":"2024-03-27T08:44:08.114526+0000","last_clean":"2024-03-27T08:44:08.114526+0000","last_became_active":"2024-03-27T08:31:05.137462+0000","last_became_peered":"2024-03-27T08:31:05.137462+0000","last_unstale":"2024-03-27T08:44:08.114526+0000","last_undegraded":"2024-03-27T08:44:08.114526+0000","last_fullsized":"2024-03-27T08:44:08.114526+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.357491+0000","last_change":"2024-03-27T08:31:05.091071+0000","last_active":"2024-03-27T08:44:09.357491+0000","last_peered":"2024-03-27T08:44:09.357491+0000","last_clean":"2024-03-27T08:44:09.357491+0000","last_became_active":"2024-03-27T08:31:05.077928+0000","last_became_peered":"2024-03-27T08:31:05.077928+0000","last_unstale":"2024-03-27T08:44:09.357491+0000","last_undegraded":"2024-03-27T08:44:09.357491+0000","last_fullsized":"2024-03-27T08:44:09.357491+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.406084+0000","last_change":"2024-03-27T08:31:05.096352+0000","last_active":"2024-03-27T08:44:09.406084+0000","last_peered":"2024-03-27T08:44:09.406084+0000","last_clean":"2024-03-27T08:44:09.406084+0000","last_became_active":"2024-03-27T08:31:05.090906+0000","last_became_peered":"2024-03-27T08:31:05.090906+0000","last_unstale":"2024-03-27T08:44:09.406084+0000","last_undegraded":"2024-03-27T08:44:09.406084+0000","last_fullsized":"2024-03-27T08:44:09.406084+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.412305+0000","last_change":"2024-03-27T08:31:05.198445+0000","last_active":"2024-03-27T08:44:09.412305+0000","last_peered":"2024-03-27T08:44:09.412305+0000","last_clean":"2024-03-27T08:44:09.412305+0000","last_became_active":"2024-03-27T08:31:05.192295+0000","last_became_peered":"2024-03-27T08:31:05.192295+0000","last_unstale":"2024-03-27T08:44:09.412305+0000","last_undegraded":"2024-03-27T08:44:09.412305+0000","last_fullsized":"2024-03-27T08:44:09.412305+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1281,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.448126+0000","last_change":"2024-03-27T08:31:05.085798+0000","last_active":"2024-03-27T08:44:09.448126+0000","last_peered":"2024-03-27T08:44:09.448126+0000","last_clean":"2024-03-27T08:44:09.448126+0000","last_became_active":"2024-03-27T08:31:05.077394+0000","last_became_peered":"2024-03-27T08:31:05.077394+0000","last_unstale":"2024-03-27T08:44:09.448126+0000","last_undegraded":"2024-03-27T08:44:09.448126+0000","last_fullsized":"2024-03-27T08:44:09.448126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.347767+0000","last_change":"2024-03-27T08:31:05.120108+0000","last_active":"2024-03-27T08:44:09.347767+0000","last_peered":"2024-03-27T08:44:09.347767+0000","last_clean":"2024-03-27T08:44:09.347767+0000","last_became_active":"2024-03-27T08:31:05.114332+0000","last_became_peered":"2024-03-27T08:31:05.114332+0000","last_unstale":"2024-03-27T08:44:09.347767+0000","last_undegraded":"2024-03-27T08:44:09.347767+0000","last_fullsized":"2024-03-27T08:44:09.347767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.074802+0000","last_change":"2024-03-27T08:31:05.181415+0000","last_active":"2024-03-27T08:44:08.074802+0000","last_peered":"2024-03-27T08:44:08.074802+0000","last_clean":"2024-03-27T08:44:08.074802+0000","last_became_active":"2024-03-27T08:31:05.175030+0000","last_became_peered":"2024-03-27T08:31:05.175030+0000","last_unstale":"2024-03-27T08:44:08.074802+0000","last_undegraded":"2024-03-27T08:44:08.074802+0000","last_fullsized":"2024-03-27T08:44:08.074802+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.105795+0000","last_change":"2024-03-27T08:31:05.169539+0000","last_active":"2024-03-27T08:44:08.105795+0000","last_peered":"2024-03-27T08:44:08.105795+0000","last_clean":"2024-03-27T08:44:08.105795+0000","last_became_active":"2024-03-27T08:31:05.164396+0000","last_became_peered":"2024-03-27T08:31:05.164396+0000","last_unstale":"2024-03-27T08:44:08.105795+0000","last_undegraded":"2024-03-27T08:44:08.105795+0000","last_fullsized":"2024-03-27T08:44:08.105795+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.585915+0000","last_change":"2024-03-27T08:31:05.116401+0000","last_active":"2024-03-27T08:44:09.585915+0000","last_peered":"2024-03-27T08:44:09.585915+0000","last_clean":"2024-03-27T08:44:09.585915+0000","last_became_active":"2024-03-27T08:31:05.098437+0000","last_became_peered":"2024-03-27T08:31:05.098437+0000","last_unstale":"2024-03-27T08:44:09.585915+0000","last_undegraded":"2024-03-27T08:44:09.585915+0000","last_fullsized":"2024-03-27T08:44:09.585915+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1255,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.105080+0000","last_change":"2024-03-27T08:31:05.141787+0000","last_active":"2024-03-27T08:44:08.105080+0000","last_peered":"2024-03-27T08:44:08.105080+0000","last_clean":"2024-03-27T08:44:08.105080+0000","last_became_active":"2024-03-27T08:31:05.120236+0000","last_became_peered":"2024-03-27T08:31:05.120236+0000","last_unstale":"2024-03-27T08:44:08.105080+0000","last_undegraded":"2024-03-27T08:44:08.105080+0000","last_fullsized":"2024-03-27T08:44:08.105080+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.566589+0000","last_change":"2024-03-27T08:31:05.209271+0000","last_active":"2024-03-27T08:44:09.566589+0000","last_peered":"2024-03-27T08:44:09.566589+0000","last_clean":"2024-03-27T08:44:09.566589+0000","last_became_active":"2024-03-27T08:31:05.203943+0000","last_became_peered":"2024-03-27T08:31:05.203943+0000","last_unstale":"2024-03-27T08:44:09.566589+0000","last_undegraded":"2024-03-27T08:44:09.566589+0000","last_fullsized":"2024-03-27T08:44:09.566589+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.545449+0000","last_change":"2024-03-27T08:31:05.095455+0000","last_active":"2024-03-27T08:44:09.545449+0000","last_peered":"2024-03-27T08:44:09.545449+0000","last_clean":"2024-03-27T08:44:09.545449+0000","last_became_active":"2024-03-27T08:31:05.072116+0000","last_became_peered":"2024-03-27T08:31:05.072116+0000","last_unstale":"2024-03-27T08:44:09.545449+0000","last_undegraded":"2024-03-27T08:44:09.545449+0000","last_fullsized":"2024-03-27T08:44:09.545449+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1249,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.585098+0000","last_change":"2024-03-27T08:31:05.158859+0000","last_active":"2024-03-27T08:44:09.585098+0000","last_peered":"2024-03-27T08:44:09.585098+0000","last_clean":"2024-03-27T08:44:09.585098+0000","last_became_active":"2024-03-27T08:31:05.151468+0000","last_became_peered":"2024-03-27T08:31:05.151468+0000","last_unstale":"2024-03-27T08:44:09.585098+0000","last_undegraded":"2024-03-27T08:44:09.585098+0000","last_fullsized":"2024-03-27T08:44:09.585098+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":1294,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.433217+0000","last_change":"2024-03-27T08:31:05.069384+0000","last_active":"2024-03-27T08:44:09.433217+0000","last_peered":"2024-03-27T08:44:09.433217+0000","last_clean":"2024-03-27T08:44:09.433217+0000","last_became_active":"2024-03-27T08:31:05.058450+0000","last_became_peered":"2024-03-27T08:31:05.058450+0000","last_unstale":"2024-03-27T08:44:09.433217+0000","last_undegraded":"2024-03-27T08:44:09.433217+0000","last_fullsized":"2024-03-27T08:44:09.433217+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1267,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.346226+0000","last_change":"2024-03-27T08:31:05.085448+0000","last_active":"2024-03-27T08:44:09.346226+0000","last_peered":"2024-03-27T08:44:09.346226+0000","last_clean":"2024-03-27T08:44:09.346226+0000","last_became_active":"2024-03-27T08:31:05.070139+0000","last_became_peered":"2024-03-27T08:31:05.070139+0000","last_unstale":"2024-03-27T08:44:09.346226+0000","last_undegraded":"2024-03-27T08:44:09.346226+0000","last_fullsized":"2024-03-27T08:44:09.346226+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.500672+0000","last_change":"2024-03-27T08:26:13.349679+0000","last_active":"2024-03-27T08:44:09.500672+0000","last_peered":"2024-03-27T08:44:09.500672+0000","last_clean":"2024-03-27T08:44:09.500672+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:44:09.500672+0000","last_undegraded":"2024-03-27T08:44:09.500672+0000","last_fullsized":"2024-03-27T08:44:09.500672+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1857,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.397617+0000","last_change":"2024-03-27T08:26:13.342923+0000","last_active":"2024-03-27T08:44:09.397617+0000","last_peered":"2024-03-27T08:44:09.397617+0000","last_clean":"2024-03-27T08:44:09.397617+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:44:09.397617+0000","last_undegraded":"2024-03-27T08:44:09.397617+0000","last_fullsized":"2024-03-27T08:44:09.397617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1861,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.152693+0000","last_change":"2024-03-27T08:26:13.346210+0000","last_active":"2024-03-27T08:44:08.152693+0000","last_peered":"2024-03-27T08:44:08.152693+0000","last_clean":"2024-03-27T08:44:08.152693+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:44:08.152693+0000","last_undegraded":"2024-03-27T08:44:08.152693+0000","last_fullsized":"2024-03-27T08:44:08.152693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1857,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.330575+0000","last_change":"2024-03-27T08:26:13.333247+0000","last_active":"2024-03-27T08:44:09.330575+0000","last_peered":"2024-03-27T08:44:09.330575+0000","last_clean":"2024-03-27T08:44:09.330575+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:44:09.330575+0000","last_undegraded":"2024-03-27T08:44:09.330575+0000","last_fullsized":"2024-03-27T08:44:09.330575+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1845,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.660928+0000","last_change":"2024-03-27T08:26:13.347337+0000","last_active":"2024-03-27T08:44:09.660928+0000","last_peered":"2024-03-27T08:44:09.660928+0000","last_clean":"2024-03-27T08:44:09.660928+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:44:09.660928+0000","last_undegraded":"2024-03-27T08:44:09.660928+0000","last_fullsized":"2024-03-27T08:44:09.660928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.430296+0000","last_change":"2024-03-27T08:26:13.336420+0000","last_active":"2024-03-27T08:44:09.430296+0000","last_peered":"2024-03-27T08:44:09.430296+0000","last_clean":"2024-03-27T08:44:09.430296+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:44:09.430296+0000","last_undegraded":"2024-03-27T08:44:09.430296+0000","last_fullsized":"2024-03-27T08:44:09.430296+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1845,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.650321+0000","last_change":"2024-03-27T08:26:13.341429+0000","last_active":"2024-03-27T08:44:09.650321+0000","last_peered":"2024-03-27T08:44:09.650321+0000","last_clean":"2024-03-27T08:44:09.650321+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:44:09.650321+0000","last_undegraded":"2024-03-27T08:44:09.650321+0000","last_fullsized":"2024-03-27T08:44:09.650321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1869,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:09.503645+0000","last_change":"2024-03-27T08:26:13.347502+0000","last_active":"2024-03-27T08:44:09.503645+0000","last_peered":"2024-03-27T08:44:09.503645+0000","last_clean":"2024-03-27T08:44:09.503645+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:44:09.503645+0000","last_undegraded":"2024-03-27T08:44:09.503645+0000","last_fullsized":"2024-03-27T08:44:09.503645+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1994,"reported_epoch":710,"state":"active+clean","last_fresh":"2024-03-27T08:44:08.086498+0000","last_change":"2024-03-27T08:26:11.311391+0000","last_active":"2024-03-27T08:44:08.086498+0000","last_peered":"2024-03-27T08:44:08.086498+0000","last_clean":"2024-03-27T08:44:08.086498+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:44:08.086498+0000","last_undegraded":"2024-03-27T08:44:08.086498+0000","last_fullsized":"2024-03-27T08:44:08.086498+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79076,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778524,"statfs":{"total":107374182400,"available":107293208576,"internally_reserved":0,"allocated":3633152,"data_stored":1464155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66592,"kb_used_data":3544,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791008,"statfs":{"total":107374182400,"available":107305992192,"internally_reserved":0,"allocated":3629056,"data_stored":1459999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83808,"kb_used_data":4120,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773792,"statfs":{"total":107374182400,"available":107288363008,"internally_reserved":0,"allocated":4218880,"data_stored":2065809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654706772,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":67104,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104790496,"statfs":{"total":107374182400,"available":107305467904,"internally_reserved":0,"allocated":4153344,"data_stored":2018170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:44:41.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:44:41.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.931+0000 7f224dd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:41.931 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.936+0000 7f224cafb640 1 Processor -- start 2024-03-27T08:44:41.931 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.936+0000 7f224cafb640 1 -- start start 2024-03-27T08:44:41.933 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.937+0000 7f224cafb640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22400b9b90 0x7f22400b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.933 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.937+0000 7f224cafb640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22400ba4e0 con 0x7f22400b6000 2024-03-27T08:44:41.933 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.937+0000 7f224cafb640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f2240022ec0 con 0x7f22400b9b90 2024-03-27T08:44:41.933 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22477fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22400b9b90 0x7f22400b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.933 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22477fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f22400b9b90 0x7f22400b9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42328/0 (socket says 172.21.15.104:42328) 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f2246ffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f22400b6000 0x7f22400b6410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43212/0 (socket says 172.21.15.104:43212) 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22477fe640 1 -- 172.21.15.104:0/2473247193 learned_addr learned my addr 172.21.15.104:0/2473247193 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22477fe640 1 -- 172.21.15.104:0/2473247193 >> v1:172.21.15.104:6789/0 conn(0x7f22400b6000 legacy=0x7f22400b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22477fe640 1 -- 172.21.15.104:0/2473247193 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2240023350 con 0x7f22400b9b90 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22477fe640 1 --2- 172.21.15.104:0/2473247193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b9b90 0x7f22400b9fa0 crc :-1 s=READY pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f4afc20cacc7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22467fc640 1 -- 172.21.15.104:0/2473247193 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f223c009e50 con 0x7f22400b9b90 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22467fc640 1 -- 172.21.15.104:0/2473247193 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f223c00e230 con 0x7f22400b9b90 2024-03-27T08:44:41.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.938+0000 7f22467fc640 1 -- 172.21.15.104:0/2473247193 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f223c00e4d0 con 0x7f22400b9b90 2024-03-27T08:44:41.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 -- 172.21.15.104:0/2473247193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b9b90 msgr2=0x7f22400b9fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:41.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 --2- 172.21.15.104:0/2473247193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b9b90 0x7f22400b9fa0 crc :-1 s=READY pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 -- 172.21.15.104:0/2473247193 shutdown_connections 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 --2- 172.21.15.104:0/2473247193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b9b90 0x7f22400b9fa0 unknown :-1 s=CLOSED pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 -- 172.21.15.104:0/2473247193 >> 172.21.15.104:0/2473247193 conn(0x7f2240029600 msgr2=0x7f2240029a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 -- 172.21.15.104:0/2473247193 shutdown_connections 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 1 -- 172.21.15.104:0/2473247193 wait complete. 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.939+0000 7f224cafb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.941+0000 7f224cafb640 1 Processor -- start 2024-03-27T08:44:41.936 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.941+0000 7f224cafb640 1 -- start start 2024-03-27T08:44:41.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f224cafb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 0x7f22400a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f224cafb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22400a3050 con 0x7f22400b6000 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f22477fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 0x7f22400a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f22477fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 0x7f22400a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42344/0 (socket says 172.21.15.104:42344) 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f22477fe640 1 -- 172.21.15.104:0/1290840815 learned_addr learned my addr 172.21.15.104:0/1290840815 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f22477fe640 1 -- 172.21.15.104:0/1290840815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22400a4ad0 con 0x7f22400b6000 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f22477fe640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 0x7f22400a1ce0 crc :-1 s=READY pgs=12991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f223c00ed20 con 0x7f22400b6000 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f223c00ee80 con 0x7f22400b6000 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22400286c0 con 0x7f22400b6000 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f223c015550 con 0x7f22400b6000 2024-03-27T08:44:41.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.942+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22400a3f80 con 0x7f22400b6000 2024-03-27T08:44:41.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.943+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f223c0156b0 con 0x7f22400b6000 2024-03-27T08:44:41.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.943+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2240004f70 con 0x7f22400b6000 2024-03-27T08:44:41.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.944+0000 7f2233fff640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f222c0387a0 0x7f222c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:41.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.944+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 397..712) ==== 3403+0+0 (crc 0 0 0) 0x7f223c0498d0 con 0x7f22400b6000 2024-03-27T08:44:41.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.944+0000 7f2246ffd640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f222c0387a0 0x7f222c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:41.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.944+0000 7f2246ffd640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f222c0387a0 0x7f222c03acb0 crc :-1 s=READY pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:41.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:41.949+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f223c010be0 con 0x7f22400b6000 2024-03-27T08:44:42.131 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.136+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22400239c0 con 0x7f22400b6000 2024-03-27T08:44:42.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.137+0000 7f2233fff640 1 -- 172.21.15.104:0/1290840815 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v712) ==== 74+0+11151 (crc 0 0 0) 0x7f223c0107d0 con 0x7f22400b6000 2024-03-27T08:44:42.132 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:44:42.132 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":712,"fsid":"756f8b0d-eb2f-4187-9610-708a3d134344","created":"2024-03-27T08:26:01.843411+0000","modified":"2024-03-27T08:44:40.087041+0000","last_up_change":"2024-03-27T08:26:08.257920+0000","last_in_change":"2024-03-27T08:26:04.020627+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":201,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:26:10.238077+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-27T08:26:12.210520+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":75,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-03-27T08:31:02.937437+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19101a14-7c59-40ec-845d-53031c7f1fa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":699,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1714017423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6805","nonce":1714017423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1714017423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6811","nonce":1714017423}]},"public_addr":"172.21.15.104:6804/1714017423","cluster_addr":"172.21.15.104:6805/1714017423","heartbeat_back_addr":"172.21.15.104:6810/1714017423","heartbeat_front_addr":"172.21.15.104:6811/1714017423","state":["exists","up"]},{"osd":1,"uuid":"e5f68525-afd8-4fe6-8c06-3eedba34bef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":699,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6801","nonce":1224298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1224298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6809","nonce":1224298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1224298329}]},"public_addr":"172.21.15.104:6801/1224298329","cluster_addr":"172.21.15.104:6802/1224298329","heartbeat_back_addr":"172.21.15.104:6809/1224298329","heartbeat_front_addr":"172.21.15.104:6808/1224298329","state":["exists","up"]},{"osd":2,"uuid":"9aa42594-3a8e-45e3-b71c-779890d7f880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":699,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6805","nonce":2552358937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2552358937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":2552358937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2552358937}]},"public_addr":"172.21.15.177:6805/2552358937","cluster_addr":"172.21.15.177:6804/2552358937","heartbeat_back_addr":"172.21.15.177:6813/2552358937","heartbeat_front_addr":"172.21.15.177:6814/2552358937","state":["exists","up"]},{"osd":3,"uuid":"032cfe60-0672-43ec-b43f-aba7bf450307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":699,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":929709834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":929709834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":929709834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":929709834}]},"public_addr":"172.21.15.177:6807/929709834","cluster_addr":"172.21.15.177:6806/929709834","heartbeat_back_addr":"172.21.15.177:6812/929709834","heartbeat_front_addr":"172.21.15.177:6811/929709834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":100,"snaps":[{"begin":1,"length":4}]},{"pool":101,"snaps":[{"begin":2,"length":4},{"begin":7,"length":2}]},{"pool":102,"snaps":[{"begin":1,"length":6}]},{"pool":103,"snaps":[{"begin":2,"length":4},{"begin":9,"length":1}]},{"pool":104,"snaps":[{"begin":2,"length":1}]},{"pool":180,"snaps":[{"begin":1,"length":1}]},{"pool":181,"snaps":[{"begin":1,"length":1}]},{"pool":183,"snaps":[{"begin":1,"length":1}]},{"pool":184,"snaps":[{"begin":1,"length":2}]},{"pool":185,"snaps":[{"begin":3,"length":1}]},{"pool":188,"snaps":[{"begin":2,"length":2}]},{"pool":190,"snaps":[{"begin":3,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.139+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f222c0387a0 msgr2=0x7f222c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.139+0000 7f224cafb640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f222c0387a0 0x7f222c03acb0 crc :-1 s=READY pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.139+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 msgr2=0x7f22400a1ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.139+0000 7f224cafb640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 0x7f22400a1ce0 crc :-1 s=READY pgs=12991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.140+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 shutdown_connections 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.140+0000 7f224cafb640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f222c0387a0 0x7f222c03acb0 unknown :-1 s=CLOSED pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.140+0000 7f224cafb640 1 --2- 172.21.15.104:0/1290840815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22400b6000 0x7f22400a1ce0 unknown :-1 s=CLOSED pgs=12991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.140+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 >> 172.21.15.104:0/1290840815 conn(0x7f2240029600 msgr2=0x7f22401416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:42.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.140+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 shutdown_connections 2024-03-27T08:44:42.136 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:42.140+0000 7f224cafb640 1 -- 172.21.15.104:0/1290840815 wait complete. 2024-03-27T08:44:43.147 INFO:tasks.ceph:Scrubbing osd.0 2024-03-27T08:44:43.147 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:44:43.273 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.270+0000 7fd5870ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.274 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.279+0000 7fd5870ad640 1 Processor -- start 2024-03-27T08:44:43.274 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.280+0000 7fd5870ad640 1 -- start start 2024-03-27T08:44:43.276 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd5870ad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd580133cc0 0x7fd5801360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.276 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd5870ad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58006f2c0 con 0x7fd58006e4f0 2024-03-27T08:44:43.276 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd5870ad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fd580130950 con 0x7fd580133cc0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57ffff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fd580133cc0 0x7fd5801360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd584e49640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fd58006e4f0 0x7fd580130f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43222/0 (socket says 172.21.15.104:43222) 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd584e49640 1 -- 172.21.15.104:0/3735024116 learned_addr learned my addr 172.21.15.104:0/3735024116 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57ffff640 1 -- 172.21.15.104:0/3735024116 >> v1:172.21.15.104:6789/0 conn(0x7fd58006e4f0 legacy=0x7fd580130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57ffff640 1 -- 172.21.15.104:0/3735024116 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd580130c60 con 0x7fd580133cc0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57f7fe640 1 -- 172.21.15.104:0/3735024116 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348955634 0 0) 0x7fd58006f2c0 con 0x7fd58006e4f0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57ffff640 1 --2- 172.21.15.104:0/3735024116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd580133cc0 0x7fd5801360e0 crc :-1 s=READY pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bafe316621d628 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57f7fe640 1 -- 172.21.15.104:0/3735024116 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd574009d90 con 0x7fd580133cc0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57f7fe640 1 -- 172.21.15.104:0/3735024116 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd57400e230 con 0x7fd580133cc0 2024-03-27T08:44:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.281+0000 7fd57f7fe640 1 -- 172.21.15.104:0/3735024116 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd57400e4d0 con 0x7fd580133cc0 2024-03-27T08:44:43.278 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 -- 172.21.15.104:0/3735024116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd580133cc0 msgr2=0x7fd5801360e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3735024116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd580133cc0 0x7fd5801360e0 crc :-1 s=READY pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 -- 172.21.15.104:0/3735024116 shutdown_connections 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3735024116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd580133cc0 0x7fd5801360e0 unknown :-1 s=CLOSED pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 -- 172.21.15.104:0/3735024116 >> 172.21.15.104:0/3735024116 conn(0x7fd58009d1d0 msgr2=0x7fd58009d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 -- 172.21.15.104:0/3735024116 shutdown_connections 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 1 -- 172.21.15.104:0/3735024116 wait complete. 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.282+0000 7fd5870ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.284+0000 7fd5870ad640 1 Processor -- start 2024-03-27T08:44:43.279 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.284+0000 7fd5870ad640 1 -- start start 2024-03-27T08:44:43.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd5870ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 0x7fd5801c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.280 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd5870ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd58011b850 con 0x7fd58006e4f0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd584e49640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 0x7fd5801c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd584e49640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 0x7fd5801c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42366/0 (socket says 172.21.15.104:42366) 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd584e49640 1 -- 172.21.15.104:0/3192694018 learned_addr learned my addr 172.21.15.104:0/3192694018 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd584e49640 1 -- 172.21.15.104:0/3192694018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58011a7e0 con 0x7fd58006e4f0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.285+0000 7fd584e49640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 0x7fd5801c6430 crc :-1 s=READY pgs=12994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.286+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd570004b80 con 0x7fd58006e4f0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.286+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd570004ce0 con 0x7fd58006e4f0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.286+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd570007120 con 0x7fd58006e4f0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.286+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5801c60a0 con 0x7fd58006e4f0 2024-03-27T08:44:43.281 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.286+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58011bde0 con 0x7fd58006e4f0 2024-03-27T08:44:43.282 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd58011c400 con 0x7fd58006e4f0 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fd5700072d0 con 0x7fd58006e4f0 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57d7fa640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd56c0387a0 0x7fd56c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7fd570048360 con 0x7fd58006e4f0 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57d7fa640 1 --2- 172.21.15.104:0/3192694018 >> v2:172.21.15.104:6804/1714017423 conn(0x7fd56c03e860 0x7fd56c040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 --> v2:172.21.15.104:6804/1714017423 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd56c0413e0 con 0x7fd56c03e860 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=712) ==== 24+0+0 (crc 0 0 0) 0x7fd57004a7d0 con 0x7fd58006e4f0 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.287+0000 7fd57ffff640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd56c0387a0 0x7fd56c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.288+0000 7fd57ffff640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd56c0387a0 0x7fd56c03acb0 crc :-1 s=READY pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.288+0000 7fd58564a640 1 --2- 172.21.15.104:0/3192694018 >> v2:172.21.15.104:6804/1714017423 conn(0x7fd56c03e860 0x7fd56c040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:44:43.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.292+0000 7fd58564a640 1 --2- 172.21.15.104:0/3192694018 >> v2:172.21.15.104:6804/1714017423 conn(0x7fd56c03e860 0x7fd56c040d50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.293+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== osd.0 v2:172.21.15.104:6804/1714017423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7fd56c0413e0 con 0x7fd56c03e860 2024-03-27T08:44:43.291 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.297+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 --> v2:172.21.15.104:6804/1714017423 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd5801c5f20 con 0x7fd56c03e860 2024-03-27T08:44:43.293 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.298+0000 7fd57d7fa640 1 -- 172.21.15.104:0/3192694018 <== osd.0 v2:172.21.15.104:6804/1714017423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7fd57800a820 con 0x7fd56c03e860 2024-03-27T08:44:43.293 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-03-27T08:44:43.293 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "" 2024-03-27T08:44:43.293 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-03-27T08:44:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.298+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 >> v2:172.21.15.104:6804/1714017423 conn(0x7fd56c03e860 msgr2=0x7fd56c040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.298+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3192694018 >> v2:172.21.15.104:6804/1714017423 conn(0x7fd56c03e860 0x7fd56c040d50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.298+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd56c0387a0 msgr2=0x7fd56c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.298+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd56c0387a0 0x7fd56c03acb0 crc :-1 s=READY pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 msgr2=0x7fd5801c6430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.294 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 0x7fd5801c6430 crc :-1 s=READY pgs=12994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 shutdown_connections 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3192694018 >> v2:172.21.15.104:6804/1714017423 conn(0x7fd56c03e860 0x7fd56c040d50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fd56c0387a0 0x7fd56c03acb0 unknown :-1 s=CLOSED pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 --2- 172.21.15.104:0/3192694018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd58006e4f0 0x7fd5801c6430 unknown :-1 s=CLOSED pgs=12994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 >> 172.21.15.104:0/3192694018 conn(0x7fd58009d1d0 msgr2=0x7fd5800a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 shutdown_connections 2024-03-27T08:44:43.295 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.299+0000 7fd5870ad640 1 -- 172.21.15.104:0/3192694018 wait complete. 2024-03-27T08:44:43.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-27T08:44:43.421 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.422+0000 7fc2cfced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.428+0000 7fc2cfced640 1 Processor -- start 2024-03-27T08:44:43.422 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.428+0000 7fc2cfced640 1 -- start start 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cfced640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc2c81314e0 0x7fc2c81318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cfced640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2c806f950 con 0x7fc2c8131e30 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cfced640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fc2c806e6a0 con 0x7fc2c81314e0 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cda89640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc2c81314e0 0x7fc2c81318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cd288640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fc2c8131e30 0x7fc2c812e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43238/0 (socket says 172.21.15.104:43238) 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cda89640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc2c81314e0 0x7fc2c81318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42376/0 (socket says 172.21.15.104:42376) 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cd288640 1 -- 172.21.15.104:0/4073788333 learned_addr learned my addr 172.21.15.104:0/4073788333 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cda89640 1 -- 172.21.15.104:0/4073788333 >> v1:172.21.15.104:6789/0 conn(0x7fc2c8131e30 legacy=0x7fc2c812e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.429+0000 7fc2cda89640 1 -- 172.21.15.104:0/4073788333 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c8075160 con 0x7fc2c81314e0 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cda89640 1 --2- 172.21.15.104:0/4073788333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c81318f0 crc :-1 s=READY pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2ae344d6b5d5be server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cca87640 1 -- 172.21.15.104:0/4073788333 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc2c4009d90 con 0x7fc2c81314e0 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cca87640 1 -- 172.21.15.104:0/4073788333 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc2c400e230 con 0x7fc2c81314e0 2024-03-27T08:44:43.425 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cca87640 1 -- 172.21.15.104:0/4073788333 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc2c400e4d0 con 0x7fc2c81314e0 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cfced640 1 -- 172.21.15.104:0/4073788333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 msgr2=0x7fc2c81318f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cfced640 1 --2- 172.21.15.104:0/4073788333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c81318f0 crc :-1 s=READY pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.430+0000 7fc2cfced640 1 -- 172.21.15.104:0/4073788333 shutdown_connections 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.431+0000 7fc2cfced640 1 --2- 172.21.15.104:0/4073788333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c81318f0 unknown :-1 s=CLOSED pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.431+0000 7fc2cfced640 1 -- 172.21.15.104:0/4073788333 >> 172.21.15.104:0/4073788333 conn(0x7fc2c80a0c40 msgr2=0x7fc2c80a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.431+0000 7fc2cfced640 1 -- 172.21.15.104:0/4073788333 shutdown_connections 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.431+0000 7fc2cfced640 1 -- 172.21.15.104:0/4073788333 wait complete. 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.431+0000 7fc2cfced640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.432+0000 7fc2cfced640 1 Processor -- start 2024-03-27T08:44:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.432+0000 7fc2cfced640 1 -- start start 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.433+0000 7fc2cfced640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c811de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.433+0000 7fc2cfced640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c8120f00 con 0x7fc2c81314e0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.433+0000 7fc2cda89640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c811de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.433+0000 7fc2cda89640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c811de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42384/0 (socket says 172.21.15.104:42384) 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.433+0000 7fc2cda89640 1 -- 172.21.15.104:0/1811549443 learned_addr learned my addr 172.21.15.104:0/1811549443 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.433+0000 7fc2cda89640 1 -- 172.21.15.104:0/1811549443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c809bf60 con 0x7fc2c81314e0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.434+0000 7fc2cda89640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c811de60 crc :-1 s=READY pgs=12997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.434+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc2c4014750 con 0x7fc2c81314e0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.434+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc2c40148b0 con 0x7fc2c81314e0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.434+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c811fc80 con 0x7fc2c81314e0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.434+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc2c4014b50 con 0x7fc2c81314e0 2024-03-27T08:44:43.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.434+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c811f2f0 con 0x7fc2c81314e0 2024-03-27T08:44:43.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.435+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c8063450 con 0x7fc2c81314e0 2024-03-27T08:44:43.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.435+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fc2c4014d00 con 0x7fc2c81314e0 2024-03-27T08:44:43.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.435+0000 7fc2be7fc640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2ac0387a0 0x7fc2ac03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.435+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7fc2c4019040 con 0x7fc2c81314e0 2024-03-27T08:44:43.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.435+0000 7fc2cd288640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2ac0387a0 0x7fc2ac03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.431 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.436+0000 7fc2cd288640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2ac0387a0 0x7fc2ac03acb0 crc :-1 s=READY pgs=3881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.440+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fc2c40185a0 con 0x7fc2c81314e0 2024-03-27T08:44:43.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.614+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc2c809ce80 con 0x7fc2ac0387a0 2024-03-27T08:44:43.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.614+0000 7fc2be7fc640 1 -- 172.21.15.104:0/1811549443 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fc2b8001250 con 0x7fc2ac0387a0 2024-03-27T08:44:43.609 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 0 to deep-scrub 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2ac0387a0 msgr2=0x7fc2ac03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2ac0387a0 0x7fc2ac03acb0 crc :-1 s=READY pgs=3881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 msgr2=0x7fc2c811de60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c811de60 crc :-1 s=READY pgs=12997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 shutdown_connections 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2ac0387a0 0x7fc2ac03acb0 unknown :-1 s=CLOSED pgs=3881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 --2- 172.21.15.104:0/1811549443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2c81314e0 0x7fc2c811de60 unknown :-1 s=CLOSED pgs=12997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 >> 172.21.15.104:0/1811549443 conn(0x7fc2c80a0c40 msgr2=0x7fc2c81b9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 shutdown_connections 2024-03-27T08:44:43.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.617+0000 7fc2cfced640 1 -- 172.21.15.104:0/1811549443 wait complete. 2024-03-27T08:44:43.628 INFO:tasks.ceph:Scrubbing osd.1 2024-03-27T08:44:43.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:44:43.746 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.744+0000 7f05bfaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.747 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.752+0000 7f05bfaed640 1 Processor -- start 2024-03-27T08:44:43.747 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.753+0000 7f05bfaed640 1 -- start start 2024-03-27T08:44:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bfaed640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f05b809f5f0 0x7f05b809fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bfaed640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80a3470 con 0x7f05b809ff40 2024-03-27T08:44:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bfaed640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f05b809f2e0 con 0x7f05b809f5f0 2024-03-27T08:44:43.749 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bd889640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f05b809f5f0 0x7f05b809fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bd088640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f05b809ff40 0x7f05b81345b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43252/0 (socket says 172.21.15.104:43252) 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bd088640 1 -- 172.21.15.104:0/744413001 learned_addr learned my addr 172.21.15.104:0/744413001 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bd889640 1 -- 172.21.15.104:0/744413001 >> v1:172.21.15.104:6789/0 conn(0x7f05b809ff40 legacy=0x7f05b81345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bd889640 1 -- 172.21.15.104:0/744413001 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b806ea10 con 0x7f05b809f5f0 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bd889640 1 --2- 172.21.15.104:0/744413001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b809fa00 crc :-1 s=READY pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246004f7a650977 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.754+0000 7f05bc887640 1 -- 172.21.15.104:0/744413001 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f05b400e040 con 0x7f05b809f5f0 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bc887640 1 -- 172.21.15.104:0/744413001 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f05b4007670 con 0x7f05b809f5f0 2024-03-27T08:44:43.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bc887640 1 -- 172.21.15.104:0/744413001 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f05b4007910 con 0x7f05b809f5f0 2024-03-27T08:44:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 -- 172.21.15.104:0/744413001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 msgr2=0x7f05b809fa00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 --2- 172.21.15.104:0/744413001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b809fa00 crc :-1 s=READY pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 -- 172.21.15.104:0/744413001 shutdown_connections 2024-03-27T08:44:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 --2- 172.21.15.104:0/744413001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b809fa00 unknown :-1 s=CLOSED pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 -- 172.21.15.104:0/744413001 >> 172.21.15.104:0/744413001 conn(0x7f05b80a15e0 msgr2=0x7f05b80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.751 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 -- 172.21.15.104:0/744413001 shutdown_connections 2024-03-27T08:44:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.755+0000 7f05bfaed640 1 -- 172.21.15.104:0/744413001 wait complete. 2024-03-27T08:44:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.756+0000 7f05bfaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.757+0000 7f05bfaed640 1 Processor -- start 2024-03-27T08:44:43.752 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.757+0000 7f05bfaed640 1 -- start start 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.758+0000 7f05bfaed640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b811dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.758+0000 7f05bfaed640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f05b811f170 con 0x7f05b809f5f0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.758+0000 7f05bd889640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b811dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.758+0000 7f05bd889640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b811dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42402/0 (socket says 172.21.15.104:42402) 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.758+0000 7f05bd889640 1 -- 172.21.15.104:0/3674712335 learned_addr learned my addr 172.21.15.104:0/3674712335 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05bd889640 1 -- 172.21.15.104:0/3674712335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8121de0 con 0x7f05b809f5f0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05bd889640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b811dc50 crc :-1 s=READY pgs=13000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f05b400e040 con 0x7f05b809f5f0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f05b4012a80 con 0x7f05b809f5f0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b8131e50 con 0x7f05b809f5f0 2024-03-27T08:44:43.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f05b4012d20 con 0x7f05b809f5f0 2024-03-27T08:44:43.755 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.759+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b8121120 con 0x7f05b809f5f0 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f05b40153f0 con 0x7f05b809f5f0 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05b8120300 con 0x7f05b809f5f0 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05ae7fc640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f059c0387a0 0x7f059c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f05b4049bb0 con 0x7f05b809f5f0 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05ae7fc640 1 --2- 172.21.15.104:0/3674712335 >> v2:172.21.15.104:6801/1224298329 conn(0x7f059c03e860 0x7f059c040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 --> v2:172.21.15.104:6801/1224298329 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f059c0413e0 con 0x7f059c03e860 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=712) ==== 24+0+0 (crc 0 0 0) 0x7f05b400fc80 con 0x7f05b809f5f0 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.760+0000 7f05bd088640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f059c0387a0 0x7f059c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.761+0000 7f05bd088640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f059c0387a0 0x7f059c03acb0 crc :-1 s=READY pgs=3882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.757 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.761+0000 7f05be08a640 1 --2- 172.21.15.104:0/3674712335 >> v2:172.21.15.104:6801/1224298329 conn(0x7f059c03e860 0x7f059c040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:44:43.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.765+0000 7f05be08a640 1 --2- 172.21.15.104:0/3674712335 >> v2:172.21.15.104:6801/1224298329 conn(0x7f059c03e860 0x7f059c040d50 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.761 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.766+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== osd.1 v2:172.21.15.104:6801/1224298329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f059c0413e0 con 0x7f059c03e860 2024-03-27T08:44:43.764 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.769+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 --> v2:172.21.15.104:6801/1224298329 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f05b8134cc0 con 0x7f059c03e860 2024-03-27T08:44:43.765 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05ae7fc640 1 -- 172.21.15.104:0/3674712335 <== osd.1 v2:172.21.15.104:6801/1224298329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f05b0009830 con 0x7f059c03e860 2024-03-27T08:44:43.766 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-03-27T08:44:43.766 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "" 2024-03-27T08:44:43.766 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-03-27T08:44:43.767 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 >> v2:172.21.15.104:6801/1224298329 conn(0x7f059c03e860 msgr2=0x7f059c040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.767 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 --2- 172.21.15.104:0/3674712335 >> v2:172.21.15.104:6801/1224298329 conn(0x7f059c03e860 0x7f059c040d50 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.767 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f059c0387a0 msgr2=0x7f059c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f059c0387a0 0x7f059c03acb0 crc :-1 s=READY pgs=3882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 msgr2=0x7f05b811dc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b811dc50 crc :-1 s=READY pgs=13000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 shutdown_connections 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f059c0387a0 0x7f059c03acb0 unknown :-1 s=CLOSED pgs=3882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 --2- 172.21.15.104:0/3674712335 >> v2:172.21.15.104:6801/1224298329 conn(0x7f059c03e860 0x7f059c040d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 --2- 172.21.15.104:0/3674712335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05b809f5f0 0x7f05b811dc50 unknown :-1 s=CLOSED pgs=13000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 >> 172.21.15.104:0/3674712335 conn(0x7f05b80a15e0 msgr2=0x7f05b81bc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.771+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 shutdown_connections 2024-03-27T08:44:43.768 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.772+0000 7f05bfaed640 1 -- 172.21.15.104:0/3674712335 wait complete. 2024-03-27T08:44:43.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-27T08:44:43.887 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.887+0000 7fa0f1d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.893+0000 7fa0f1d9d640 1 Processor -- start 2024-03-27T08:44:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.893+0000 7fa0f1d9d640 1 -- start start 2024-03-27T08:44:43.889 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0f1d9d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa0ec12e2c0 0x7fa0ec12e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0f1d9d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0ec06e090 con 0x7fa0ec131320 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0f1d9d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec09ef70 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0eb7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa0ec12e2c0 0x7fa0ec12e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0eaffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fa0ec131320 0x7fa0ec131730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43264/0 (socket says 172.21.15.104:43264) 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0eaffd640 1 -- 172.21.15.104:0/302014522 learned_addr learned my addr 172.21.15.104:0/302014522 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.894+0000 7fa0eb7fe640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fa0ec12e2c0 0x7fa0ec12e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42418/0 (socket says 172.21.15.104:42418) 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.895+0000 7fa0eb7fe640 1 -- 172.21.15.104:0/302014522 >> v1:172.21.15.104:6789/0 conn(0x7fa0ec131320 legacy=0x7fa0ec131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.895+0000 7fa0eb7fe640 1 -- 172.21.15.104:0/302014522 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0734e0 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.895+0000 7fa0eb7fe640 1 --2- 172.21.15.104:0/302014522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec12e6d0 crc :-1 s=READY pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a162e114410923 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.895+0000 7fa0ea7fc640 1 -- 172.21.15.104:0/302014522 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0dc009d90 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.890 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.895+0000 7fa0ea7fc640 1 -- 172.21.15.104:0/302014522 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa0dc00e230 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.891 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.895+0000 7fa0ea7fc640 1 -- 172.21.15.104:0/302014522 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0dc00e4d0 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/302014522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 msgr2=0x7fa0ec12e6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 --2- 172.21.15.104:0/302014522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec12e6d0 crc :-1 s=READY pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/302014522 shutdown_connections 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 --2- 172.21.15.104:0/302014522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec12e6d0 unknown :-1 s=CLOSED pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/302014522 >> 172.21.15.104:0/302014522 conn(0x7fa0ec09d1d0 msgr2=0x7fa0ec09d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/302014522 shutdown_connections 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/302014522 wait complete. 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.896+0000 7fa0f1d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.897+0000 7fa0f1d9d640 1 Processor -- start 2024-03-27T08:44:43.892 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.897+0000 7fa0f1d9d640 1 -- start start 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.898+0000 7fa0f1d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec119a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.898+0000 7fa0f1d9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec11d0d0 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.898+0000 7fa0eb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec119a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.898+0000 7fa0eb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec119a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42426/0 (socket says 172.21.15.104:42426) 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.898+0000 7fa0eb7fe640 1 -- 172.21.15.104:0/1486441600 learned_addr learned my addr 172.21.15.104:0/1486441600 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0eb7fe640 1 -- 172.21.15.104:0/1486441600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec12dd70 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0eb7fe640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec119a20 crc :-1 s=READY pgs=13003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0dc018040 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.894 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa0dc012650 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0dc0128f0 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec06aa50 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.899+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec11b5f0 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.895 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.900+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fa0dc01b040 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.900+0000 7fa0d3fff640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa0d80387a0 0x7fa0d803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.900+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7fa0dc04b060 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.900+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec06aa50 con 0x7fa0ec12e2c0 2024-03-27T08:44:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.900+0000 7fa0eaffd640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa0d80387a0 0x7fa0d803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:43.896 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.901+0000 7fa0eaffd640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa0d80387a0 0x7fa0d803acb0 crc :-1 s=READY pgs=3883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:43.900 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:43.906+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fa0ec06aa50 con 0x7fa0ec12e2c0 2024-03-27T08:44:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.082+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa0ec072e80 con 0x7fa0d80387a0 2024-03-27T08:44:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.083+0000 7fa0d3fff640 1 -- 172.21.15.104:0/1486441600 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fa0ec072e80 con 0x7fa0d80387a0 2024-03-27T08:44:44.078 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 1 to deep-scrub 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.086+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa0d80387a0 msgr2=0x7fa0d803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.086+0000 7fa0f1d9d640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa0d80387a0 0x7fa0d803acb0 crc :-1 s=READY pgs=3883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.086+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 msgr2=0x7fa0ec119a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.086+0000 7fa0f1d9d640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec119a20 crc :-1 s=READY pgs=13003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.087+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 shutdown_connections 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.087+0000 7fa0f1d9d640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fa0d80387a0 0x7fa0d803acb0 unknown :-1 s=CLOSED pgs=3883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.087+0000 7fa0f1d9d640 1 --2- 172.21.15.104:0/1486441600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0ec12e2c0 0x7fa0ec119a20 unknown :-1 s=CLOSED pgs=13003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.087+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 >> 172.21.15.104:0/1486441600 conn(0x7fa0ec09d1d0 msgr2=0x7fa0ec1b9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.087+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 shutdown_connections 2024-03-27T08:44:44.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.087+0000 7fa0f1d9d640 1 -- 172.21.15.104:0/1486441600 wait complete. 2024-03-27T08:44:44.100 INFO:tasks.ceph:Scrubbing osd.2 2024-03-27T08:44:44.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:44:44.216 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.213+0000 7f631895d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.216 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.222+0000 7f631895d640 1 Processor -- start 2024-03-27T08:44:44.217 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.222+0000 7f631895d640 1 -- start start 2024-03-27T08:44:44.219 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f631895d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f6314133cc0 0x7f63141360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f631895d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631406f2c0 con 0x7f631406e4f0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f631895d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f6314130950 con 0x7f6314133cc0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f631259c640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f631406e4f0 0x7f6314130f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43270/0 (socket says 172.21.15.104:43270) 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f631259c640 1 -- 172.21.15.104:0/408190637 learned_addr learned my addr 172.21.15.104:0/408190637 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f630bfff640 1 --2- 172.21.15.104:0/408190637 >> v2:172.21.15.104:3300/0 conn(0x7f6314133cc0 0x7f63141360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f6311d9b640 1 -- 172.21.15.104:0/408190637 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118999467 0 0) 0x7f631406f2c0 con 0x7f631406e4f0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f6311d9b640 1 -- 172.21.15.104:0/408190637 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6300003580 con 0x7f631406e4f0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f630bfff640 1 -- 172.21.15.104:0/408190637 >> v1:172.21.15.104:6789/0 conn(0x7f631406e4f0 legacy=0x7f6314130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.223+0000 7f630bfff640 1 -- 172.21.15.104:0/408190637 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6314130c60 con 0x7f6314133cc0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f630bfff640 1 --2- 172.21.15.104:0/408190637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6314133cc0 0x7f63141360e0 crc :-1 s=READY pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b5299d18aa1bd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f6311d9b640 1 -- 172.21.15.104:0/408190637 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f62fc00ad20 con 0x7f6314133cc0 2024-03-27T08:44:44.220 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f6311d9b640 1 -- 172.21.15.104:0/408190637 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f62fc00ae80 con 0x7f6314133cc0 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f6311d9b640 1 -- 172.21.15.104:0/408190637 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f62fc00f470 con 0x7f6314133cc0 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f631895d640 1 -- 172.21.15.104:0/408190637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6314133cc0 msgr2=0x7f63141360e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f631895d640 1 --2- 172.21.15.104:0/408190637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6314133cc0 0x7f63141360e0 crc :-1 s=READY pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f631895d640 1 -- 172.21.15.104:0/408190637 shutdown_connections 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f631895d640 1 --2- 172.21.15.104:0/408190637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6314133cc0 0x7f63141360e0 unknown :-1 s=CLOSED pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f631895d640 1 -- 172.21.15.104:0/408190637 >> 172.21.15.104:0/408190637 conn(0x7f631409d1d0 msgr2=0x7f631409d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.224+0000 7f631895d640 1 -- 172.21.15.104:0/408190637 shutdown_connections 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.225+0000 7f631895d640 1 -- 172.21.15.104:0/408190637 wait complete. 2024-03-27T08:44:44.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.225+0000 7f631895d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.226+0000 7f631895d640 1 Processor -- start 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.226+0000 7f631895d640 1 -- start start 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.227+0000 7f631895d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 0x7f63140726a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.227+0000 7f631895d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f631411c0d0 con 0x7f631406e4f0 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.227+0000 7f631259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 0x7f63140726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.227+0000 7f631259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 0x7f63140726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42448/0 (socket says 172.21.15.104:42448) 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.227+0000 7f631259c640 1 -- 172.21.15.104:0/2740554661 learned_addr learned my addr 172.21.15.104:0/2740554661 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:44.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f631259c640 1 -- 172.21.15.104:0/2740554661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6314072410 con 0x7f631406e4f0 2024-03-27T08:44:44.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f631259c640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 0x7f63140726a0 crc :-1 s=READY pgs=13006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6304005a60 con 0x7f631406e4f0 2024-03-27T08:44:44.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6304005bc0 con 0x7f631406e4f0 2024-03-27T08:44:44.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631411bb20 con 0x7f631406e4f0 2024-03-27T08:44:44.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6304005e60 con 0x7f631406e4f0 2024-03-27T08:44:44.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.228+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631411d610 con 0x7f631406e4f0 2024-03-27T08:44:44.225 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6314072210 con 0x7f631406e4f0 2024-03-27T08:44:44.225 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f630400c770 con 0x7f631406e4f0 2024-03-27T08:44:44.225 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630affd640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f62f40387a0 0x7f62f403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.225 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f63040481f0 con 0x7f631406e4f0 2024-03-27T08:44:44.225 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630affd640 1 --2- 172.21.15.104:0/2740554661 >> v2:172.21.15.177:6805/2552358937 conn(0x7f62f403e8c0 0x7f62f4040db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.225 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 --> v2:172.21.15.177:6805/2552358937 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62f4041440 con 0x7f62f403e8c0 2024-03-27T08:44:44.226 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=712) ==== 24+0+0 (crc 0 0 0) 0x7f630404a6e0 con 0x7f631406e4f0 2024-03-27T08:44:44.226 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.229+0000 7f630bfff640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f62f40387a0 0x7f62f403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.226 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.230+0000 7f630bfff640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f62f40387a0 0x7f62f403acb0 crc :-1 s=READY pgs=3884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.226 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.231+0000 7f6312d9d640 1 --2- 172.21.15.104:0/2740554661 >> v2:172.21.15.177:6805/2552358937 conn(0x7f62f403e8c0 0x7f62f4040db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:44:44.229 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.235+0000 7f6312d9d640 1 --2- 172.21.15.104:0/2740554661 >> v2:172.21.15.177:6805/2552358937 conn(0x7f62f403e8c0 0x7f62f4040db0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.231 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.237+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== osd.2 v2:172.21.15.177:6805/2552358937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f62f4041440 con 0x7f62f403e8c0 2024-03-27T08:44:44.234 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.240+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 --> v2:172.21.15.177:6805/2552358937 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f631412ffd0 con 0x7f62f403e8c0 2024-03-27T08:44:44.236 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f630affd640 1 -- 172.21.15.104:0/2740554661 <== osd.2 v2:172.21.15.177:6805/2552358937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f631412ffd0 con 0x7f62f403e8c0 2024-03-27T08:44:44.237 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-03-27T08:44:44.237 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "" 2024-03-27T08:44:44.237 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 >> v2:172.21.15.177:6805/2552358937 conn(0x7f62f403e8c0 msgr2=0x7f62f4040db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 --2- 172.21.15.104:0/2740554661 >> v2:172.21.15.177:6805/2552358937 conn(0x7f62f403e8c0 0x7f62f4040db0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f62f40387a0 msgr2=0x7f62f403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f62f40387a0 0x7f62f403acb0 crc :-1 s=READY pgs=3884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 msgr2=0x7f63140726a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 0x7f63140726a0 crc :-1 s=READY pgs=13006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 shutdown_connections 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 --2- 172.21.15.104:0/2740554661 >> v2:172.21.15.177:6805/2552358937 conn(0x7f62f403e8c0 0x7f62f4040db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.238 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f62f40387a0 0x7f62f403acb0 unknown :-1 s=CLOSED pgs=3884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.239 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 --2- 172.21.15.104:0/2740554661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f631406e4f0 0x7f63140726a0 unknown :-1 s=CLOSED pgs=13006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.239 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 >> 172.21.15.104:0/2740554661 conn(0x7f631409d1d0 msgr2=0x7f631409b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.239 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 shutdown_connections 2024-03-27T08:44:44.239 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.242+0000 7f631895d640 1 -- 172.21.15.104:0/2740554661 wait complete. 2024-03-27T08:44:44.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-27T08:44:44.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.358+0000 7f3a9775d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.363+0000 7f3a9775d640 1 Processor -- start 2024-03-27T08:44:44.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.364+0000 7f3a9775d640 1 -- start start 2024-03-27T08:44:44.360 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a9775d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3a9009a420 0x7f3a9009a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.360 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a9775d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a90099ce0 con 0x7f3a9009ad70 2024-03-27T08:44:44.360 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a9775d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f3a90099f40 con 0x7f3a9009a420 2024-03-27T08:44:44.360 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a954f9640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f3a9009a420 0x7f3a9009a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.360 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a94cf8640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f3a9009ad70 0x7f3a901390d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43278/0 (socket says 172.21.15.104:43278) 2024-03-27T08:44:44.360 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a94cf8640 1 -- 172.21.15.104:0/4001077257 learned_addr learned my addr 172.21.15.104:0/4001077257 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a954f9640 1 -- 172.21.15.104:0/4001077257 >> v1:172.21.15.104:6789/0 conn(0x7f3a9009ad70 legacy=0x7f3a901390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a954f9640 1 -- 172.21.15.104:0/4001077257 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9006b950 con 0x7f3a9009a420 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a8ffff640 1 -- 172.21.15.104:0/4001077257 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652182248 0 0) 0x7f3a90099ce0 con 0x7f3a9009ad70 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a954f9640 1 --2- 172.21.15.104:0/4001077257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a9009a830 crc :-1 s=READY pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8766fbd2da2c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a8ffff640 1 -- 172.21.15.104:0/4001077257 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3a8800e040 con 0x7f3a9009a420 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a8ffff640 1 -- 172.21.15.104:0/4001077257 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3a88007670 con 0x7f3a9009a420 2024-03-27T08:44:44.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.365+0000 7f3a8ffff640 1 -- 172.21.15.104:0/4001077257 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3a88007910 con 0x7f3a9009a420 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 -- 172.21.15.104:0/4001077257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 msgr2=0x7f3a9009a830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 --2- 172.21.15.104:0/4001077257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a9009a830 crc :-1 s=READY pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 -- 172.21.15.104:0/4001077257 shutdown_connections 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 --2- 172.21.15.104:0/4001077257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a9009a830 unknown :-1 s=CLOSED pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 -- 172.21.15.104:0/4001077257 >> 172.21.15.104:0/4001077257 conn(0x7f3a900a0e20 msgr2=0x7f3a900a1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 -- 172.21.15.104:0/4001077257 shutdown_connections 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 1 -- 172.21.15.104:0/4001077257 wait complete. 2024-03-27T08:44:44.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.366+0000 7f3a9775d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.368+0000 7f3a9775d640 1 Processor -- start 2024-03-27T08:44:44.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.368+0000 7f3a9775d640 1 -- start start 2024-03-27T08:44:44.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a9775d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a901ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a9775d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90097560 con 0x7f3a9009a420 2024-03-27T08:44:44.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a954f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a901ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a954f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a901ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42462/0 (socket says 172.21.15.104:42462) 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a954f9640 1 -- 172.21.15.104:0/3416745748 learned_addr learned my addr 172.21.15.104:0/3416745748 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a954f9640 1 -- 172.21.15.104:0/3416745748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90097cc0 con 0x7f3a9009a420 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a954f9640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a901ad170 crc :-1 s=READY pgs=13009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3a8800e040 con 0x7f3a9009a420 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3a88013aa0 con 0x7f3a9009a420 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a90098570 con 0x7f3a9009a420 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.369+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3a88013d40 con 0x7f3a9009a420 2024-03-27T08:44:44.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.370+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a901accf0 con 0x7f3a9009a420 2024-03-27T08:44:44.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.370+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f3a88011040 con 0x7f3a9009a420 2024-03-27T08:44:44.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.370+0000 7f3a8dffb640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3a7c0387a0 0x7f3a7c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.370+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a901ad7f0 con 0x7f3a9009a420 2024-03-27T08:44:44.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.371+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f3a88049980 con 0x7f3a9009a420 2024-03-27T08:44:44.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.371+0000 7f3a94cf8640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3a7c0387a0 0x7f3a7c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.371+0000 7f3a94cf8640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3a7c0387a0 0x7f3a7c03acb0 crc :-1 s=READY pgs=3885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.376+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f3a88015390 con 0x7f3a9009a420 2024-03-27T08:44:44.544 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.550+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3a901ada20 con 0x7f3a7c0387a0 2024-03-27T08:44:44.545 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.550+0000 7f3a8dffb640 1 -- 172.21.15.104:0/3416745748 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f3a84005930 con 0x7f3a7c0387a0 2024-03-27T08:44:44.545 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 2 to deep-scrub 2024-03-27T08:44:44.547 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.552+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3a7c0387a0 msgr2=0x7f3a7c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.552+0000 7f3a9775d640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3a7c0387a0 0x7f3a7c03acb0 crc :-1 s=READY pgs=3885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.552+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 msgr2=0x7f3a901ad170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.552+0000 7f3a9775d640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a901ad170 crc :-1 s=READY pgs=13009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.553+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 shutdown_connections 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.553+0000 7f3a9775d640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f3a7c0387a0 0x7f3a7c03acb0 unknown :-1 s=CLOSED pgs=3885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.553+0000 7f3a9775d640 1 --2- 172.21.15.104:0/3416745748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a9009a420 0x7f3a901ad170 unknown :-1 s=CLOSED pgs=13009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.553+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 >> 172.21.15.104:0/3416745748 conn(0x7f3a900a0e20 msgr2=0x7f3a90071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.553+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 shutdown_connections 2024-03-27T08:44:44.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.553+0000 7f3a9775d640 1 -- 172.21.15.104:0/3416745748 wait complete. 2024-03-27T08:44:44.559 INFO:tasks.ceph:Scrubbing osd.3 2024-03-27T08:44:44.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:44:44.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.671+0000 7f41052ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.680+0000 7f41052ad640 1 Processor -- start 2024-03-27T08:44:44.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.680+0000 7f41052ad640 1 -- start start 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f41052ad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4100137800 0x7f4100139c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f41052ad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f410006fc60 con 0x7f4100134850 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f41052ad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f41001344d0 con 0x7f4100137800 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f40f67fc640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4100137800 0x7f4100139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f40feffd640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f4100134850 0x7f4100134c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43294/0 (socket says 172.21.15.104:43294) 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f40feffd640 1 -- 172.21.15.104:0/965919141 learned_addr learned my addr 172.21.15.104:0/965919141 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.681+0000 7f40f67fc640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4100137800 0x7f4100139c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42470/0 (socket says 172.21.15.104:42470) 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.682+0000 7f40f67fc640 1 -- 172.21.15.104:0/965919141 >> v1:172.21.15.104:6789/0 conn(0x7f4100134850 legacy=0x7f4100134c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.682+0000 7f40f67fc640 1 -- 172.21.15.104:0/965919141 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100070b90 con 0x7f4100137800 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.682+0000 7f40f67fc640 1 --2- 172.21.15.104:0/965919141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100137800 0x7f4100139c20 crc :-1 s=READY pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f79d995f393167c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.682+0000 7f40fe7fc640 1 -- 172.21.15.104:0/965919141 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f40e8009d90 con 0x7f4100137800 2024-03-27T08:44:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.682+0000 7f40fe7fc640 1 -- 172.21.15.104:0/965919141 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f40e800e230 con 0x7f4100137800 2024-03-27T08:44:44.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.682+0000 7f40fe7fc640 1 -- 172.21.15.104:0/965919141 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f40e800e4d0 con 0x7f4100137800 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 -- 172.21.15.104:0/965919141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100137800 msgr2=0x7f4100139c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 --2- 172.21.15.104:0/965919141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100137800 0x7f4100139c20 crc :-1 s=READY pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 -- 172.21.15.104:0/965919141 shutdown_connections 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 --2- 172.21.15.104:0/965919141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100137800 0x7f4100139c20 unknown :-1 s=CLOSED pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 -- 172.21.15.104:0/965919141 >> 172.21.15.104:0/965919141 conn(0x7f41000a52b0 msgr2=0x7f41000a1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 -- 172.21.15.104:0/965919141 shutdown_connections 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 1 -- 172.21.15.104:0/965919141 wait complete. 2024-03-27T08:44:44.679 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.683+0000 7f41052ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.684+0000 7f41052ad640 1 Processor -- start 2024-03-27T08:44:44.680 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.684+0000 7f41052ad640 1 -- start start 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.685+0000 7f41052ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 0x7f41001c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.685+0000 7f41052ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f410011f030 con 0x7f4100134850 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40feffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 0x7f41001c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40feffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 0x7f41001c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42486/0 (socket says 172.21.15.104:42486) 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40feffd640 1 -- 172.21.15.104:0/512054782 learned_addr learned my addr 172.21.15.104:0/512054782 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40feffd640 1 -- 172.21.15.104:0/512054782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100131e90 con 0x7f4100134850 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40feffd640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 0x7f41001c9c00 crc :-1 s=READY pgs=13012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.681 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f40f0005a60 con 0x7f4100134850 2024-03-27T08:44:44.682 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f40f0005bc0 con 0x7f4100134850 2024-03-27T08:44:44.682 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4100121e10 con 0x7f4100134850 2024-03-27T08:44:44.682 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f40f0005e60 con 0x7f4100134850 2024-03-27T08:44:44.682 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.686+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4100121200 con 0x7f4100134850 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4100132d20 con 0x7f4100134850 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f40f000c780 con 0x7f4100134850 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f40f7fff640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f40e00387a0 0x7f40e003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f40f0048190 con 0x7f4100134850 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f40f7fff640 1 --2- 172.21.15.104:0/512054782 >> v2:172.21.15.177:6807/929709834 conn(0x7f40e003e860 0x7f40e0040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 --> v2:172.21.15.177:6807/929709834 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40e00413e0 con 0x7f40e003e860 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.687+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=712) ==== 24+0+0 (crc 0 0 0) 0x7f40f004a680 con 0x7f4100134850 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.688+0000 7f40f67fc640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f40e00387a0 0x7f40e003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.683 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.688+0000 7f40f67fc640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f40e00387a0 0x7f40e003acb0 crc :-1 s=READY pgs=3886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.684 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.689+0000 7f40ff7fe640 1 --2- 172.21.15.104:0/512054782 >> v2:172.21.15.177:6807/929709834 conn(0x7f40e003e860 0x7f40e0040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:44:44.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.693+0000 7f40ff7fe640 1 --2- 172.21.15.104:0/512054782 >> v2:172.21.15.177:6807/929709834 conn(0x7f40e003e860 0x7f40e0040d50 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.695+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== osd.3 v2:172.21.15.177:6807/929709834 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f40e00413e0 con 0x7f40e003e860 2024-03-27T08:44:44.692 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.698+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 --> v2:172.21.15.177:6807/929709834 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4100137a50 con 0x7f40e003e860 2024-03-27T08:44:44.699 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f40f7fff640 1 -- 172.21.15.104:0/512054782 <== osd.3 v2:172.21.15.177:6807/929709834 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f4100137a50 con 0x7f40e003e860 2024-03-27T08:44:44.699 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-03-27T08:44:44.700 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "" 2024-03-27T08:44:44.700 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-03-27T08:44:44.700 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 >> v2:172.21.15.177:6807/929709834 conn(0x7f40e003e860 msgr2=0x7f40e0040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 --2- 172.21.15.104:0/512054782 >> v2:172.21.15.177:6807/929709834 conn(0x7f40e003e860 0x7f40e0040d50 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f40e00387a0 msgr2=0x7f40e003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f40e00387a0 0x7f40e003acb0 crc :-1 s=READY pgs=3886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 msgr2=0x7f41001c9c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 0x7f41001c9c00 crc :-1 s=READY pgs=13012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 shutdown_connections 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 --2- 172.21.15.104:0/512054782 >> v2:172.21.15.177:6807/929709834 conn(0x7f40e003e860 0x7f40e0040d50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f40e00387a0 0x7f40e003acb0 unknown :-1 s=CLOSED pgs=3886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 --2- 172.21.15.104:0/512054782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100134850 0x7f41001c9c00 unknown :-1 s=CLOSED pgs=13012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 >> 172.21.15.104:0/512054782 conn(0x7f41000a52b0 msgr2=0x7f41001bc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 shutdown_connections 2024-03-27T08:44:44.701 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.705+0000 7f41052ad640 1 -- 172.21.15.104:0/512054782 wait complete. 2024-03-27T08:44:44.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-27T08:44:44.821 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.821+0000 7f464ccad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.827+0000 7f464ccad640 1 Processor -- start 2024-03-27T08:44:44.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.827+0000 7f464ccad640 1 -- start start 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.828+0000 7f464ccad640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f4648135a40 0x7f4648137e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.828+0000 7f464ccad640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464806bbb0 con 0x7f464812f5e0 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.828+0000 7f464ccad640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f46480735e0 con 0x7f4648135a40 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.828+0000 7f464659c640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f464812f5e0 0x7f464812f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43306/0 (socket says 172.21.15.104:43306) 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.828+0000 7f464659c640 1 -- 172.21.15.104:0/2704027045 learned_addr learned my addr 172.21.15.104:0/2704027045 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.828+0000 7f4645d9b640 1 --2- 172.21.15.104:0/2704027045 >> v2:172.21.15.104:3300/0 conn(0x7f4648135a40 0x7f4648137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f464559a640 1 -- 172.21.15.104:0/2704027045 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157410688 0 0) 0x7f464806bbb0 con 0x7f464812f5e0 2024-03-27T08:44:44.824 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f464559a640 1 -- 172.21.15.104:0/2704027045 --> v1:172.21.15.104:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4630003580 con 0x7f464812f5e0 2024-03-27T08:44:44.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f4645d9b640 1 -- 172.21.15.104:0/2704027045 >> v1:172.21.15.104:6789/0 conn(0x7f464812f5e0 legacy=0x7f464812f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f4645d9b640 1 -- 172.21.15.104:0/2704027045 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4648130100 con 0x7f4648135a40 2024-03-27T08:44:44.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f4645d9b640 1 --2- 172.21.15.104:0/2704027045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4648135a40 0x7f4648137e60 crc :-1 s=READY pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4b747aa06b0a9e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f464559a640 1 -- 172.21.15.104:0/2704027045 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f463c009d90 con 0x7f4648135a40 2024-03-27T08:44:44.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f464559a640 1 -- 172.21.15.104:0/2704027045 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f463c00e230 con 0x7f4648135a40 2024-03-27T08:44:44.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.829+0000 7f464559a640 1 -- 172.21.15.104:0/2704027045 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f463c00e4d0 con 0x7f4648135a40 2024-03-27T08:44:44.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 -- 172.21.15.104:0/2704027045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4648135a40 msgr2=0x7f4648137e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:44.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 --2- 172.21.15.104:0/2704027045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4648135a40 0x7f4648137e60 crc :-1 s=READY pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 -- 172.21.15.104:0/2704027045 shutdown_connections 2024-03-27T08:44:44.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 --2- 172.21.15.104:0/2704027045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4648135a40 0x7f4648137e60 unknown :-1 s=CLOSED pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:44.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 -- 172.21.15.104:0/2704027045 >> 172.21.15.104:0/2704027045 conn(0x7f46480a0e20 msgr2=0x7f46480a1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:44.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 -- 172.21.15.104:0/2704027045 shutdown_connections 2024-03-27T08:44:44.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 1 -- 172.21.15.104:0/2704027045 wait complete. 2024-03-27T08:44:44.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.830+0000 7f464ccad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:44.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.831+0000 7f464ccad640 1 Processor -- start 2024-03-27T08:44:44.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.832+0000 7f464ccad640 1 -- start start 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464ccad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 0x7f4648127fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464ccad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f464812b340 con 0x7f464812f5e0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 0x7f4648127fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 0x7f4648127fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42506/0 (socket says 172.21.15.104:42506) 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464659c640 1 -- 172.21.15.104:0/2635453746 learned_addr learned my addr 172.21.15.104:0/2635453746 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464659c640 1 -- 172.21.15.104:0/2635453746 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4648073c10 con 0x7f464812f5e0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464659c640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 0x7f4648127fe0 crc :-1 s=READY pgs=13015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4638005a60 con 0x7f464812f5e0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4638005bc0 con 0x7f464812f5e0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4648129860 con 0x7f464812f5e0 2024-03-27T08:44:44.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4638005e60 con 0x7f464812f5e0 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.833+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464812bce0 con 0x7f464812f5e0 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.834+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f46380070b0 con 0x7f464812f5e0 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.834+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464812a280 con 0x7f464812f5e0 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.834+0000 7f4636ffd640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f46280387a0 0x7f462803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.834+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f4638047ff0 con 0x7f464812f5e0 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.835+0000 7f4645d9b640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f46280387a0 0x7f462803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:44.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.835+0000 7f4645d9b640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f46280387a0 0x7f462803acb0 crc :-1 s=READY pgs=3887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:44.834 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:44.839+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f464812a280 con 0x7f464812f5e0 2024-03-27T08:44:45.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.016+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f464812f2d0 con 0x7f46280387a0 2024-03-27T08:44:45.011 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.017+0000 7f4636ffd640 1 -- 172.21.15.104:0/2635453746 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f463c009c60 con 0x7f46280387a0 2024-03-27T08:44:45.012 INFO:teuthology.orchestra.run.smithi104.stderr:instructed osd(s) 3 to deep-scrub 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f46280387a0 msgr2=0x7f462803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f46280387a0 0x7f462803acb0 crc :-1 s=READY pgs=3887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 msgr2=0x7f4648127fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 0x7f4648127fe0 crc :-1 s=READY pgs=13015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 shutdown_connections 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f46280387a0 0x7f462803acb0 unknown :-1 s=CLOSED pgs=3887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 --2- 172.21.15.104:0/2635453746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464812f5e0 0x7f4648127fe0 unknown :-1 s=CLOSED pgs=13015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 >> 172.21.15.104:0/2635453746 conn(0x7f46480a0e20 msgr2=0x7f464812db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:45.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 shutdown_connections 2024-03-27T08:44:45.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.019+0000 7f464ccad640 1 -- 172.21.15.104:0/2635453746 wait complete. 2024-03-27T08:44:45.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:44:45.134 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.136+0000 7f0a1806d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:45.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.140+0000 7f0a1806d640 1 Processor -- start 2024-03-27T08:44:45.135 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.141+0000 7f0a1806d640 1 -- start start 2024-03-27T08:44:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a1806d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0a1009b250 0x7f0a1009b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a1806d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a10075160 con 0x7f0a101324f0 2024-03-27T08:44:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a1806d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f0a100685f0 con 0x7f0a1009b250 2024-03-27T08:44:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15e09640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0a1009b250 0x7f0a1009b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15608640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7f0a101324f0 0x7f0a10132900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:43318/0 (socket says 172.21.15.104:43318) 2024-03-27T08:44:45.138 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15e09640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f0a1009b250 0x7f0a1009b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42516/0 (socket says 172.21.15.104:42516) 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15608640 1 -- 172.21.15.104:0/3376812700 learned_addr learned my addr 172.21.15.104:0/3376812700 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15e09640 1 -- 172.21.15.104:0/3376812700 >> v1:172.21.15.104:6789/0 conn(0x7f0a101324f0 legacy=0x7f0a10132900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15e09640 1 -- 172.21.15.104:0/3376812700 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a100709c0 con 0x7f0a1009b250 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a14e07640 1 -- 172.21.15.104:0/3376812700 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296802584 0 0) 0x7f0a10075160 con 0x7f0a101324f0 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a15e09640 1 --2- 172.21.15.104:0/3376812700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a1009b660 crc :-1 s=READY pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3836952fae1f6cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a14e07640 1 -- 172.21.15.104:0/3376812700 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0a0c009d90 con 0x7f0a1009b250 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a14e07640 1 -- 172.21.15.104:0/3376812700 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0a0c00e230 con 0x7f0a1009b250 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.142+0000 7f0a14e07640 1 -- 172.21.15.104:0/3376812700 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0a0c00e4d0 con 0x7f0a1009b250 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 -- 172.21.15.104:0/3376812700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 msgr2=0x7f0a1009b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 --2- 172.21.15.104:0/3376812700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a1009b660 crc :-1 s=READY pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 -- 172.21.15.104:0/3376812700 shutdown_connections 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 --2- 172.21.15.104:0/3376812700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a1009b660 unknown :-1 s=CLOSED pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 -- 172.21.15.104:0/3376812700 >> 172.21.15.104:0/3376812700 conn(0x7f0a100a0c40 msgr2=0x7f0a100a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 -- 172.21.15.104:0/3376812700 shutdown_connections 2024-03-27T08:44:45.139 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 1 -- 172.21.15.104:0/3376812700 wait complete. 2024-03-27T08:44:45.140 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.143+0000 7f0a1806d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:44:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.145+0000 7f0a1806d640 1 Processor -- start 2024-03-27T08:44:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.145+0000 7f0a1806d640 1 -- start start 2024-03-27T08:44:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a1806d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a101c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a1806d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10121500 con 0x7f0a1009b250 2024-03-27T08:44:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a15e09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a101c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:45.141 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a15e09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a101c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42518/0 (socket says 172.21.15.104:42518) 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a15e09640 1 -- 172.21.15.104:0/1903836221 learned_addr learned my addr 172.21.15.104:0/1903836221 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a15e09640 1 -- 172.21.15.104:0/1903836221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a101241a0 con 0x7f0a1009b250 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a15e09640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a101c75f0 crc :-1 s=READY pgs=13018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0a0c014620 con 0x7f0a1009b250 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0a0c014780 con 0x7f0a1009b250 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a10122a50 con 0x7f0a1009b250 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0a0c014a20 con 0x7f0a1009b250 2024-03-27T08:44:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.146+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a10123650 con 0x7f0a1009b250 2024-03-27T08:44:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.147+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f0a0c014bd0 con 0x7f0a1009b250 2024-03-27T08:44:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.147+0000 7f0a067fc640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f09f40387a0 0x7f09f403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:44:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.147+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f0a0c019040 con 0x7f0a1009b250 2024-03-27T08:44:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.148+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a10063450 con 0x7f0a1009b250 2024-03-27T08:44:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.148+0000 7f0a15608640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f09f40387a0 0x7f09f403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:44:45.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.148+0000 7f0a15608640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f09f40387a0 0x7f09f403acb0 crc :-1 s=READY pgs=3888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:44:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.153+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f0a0c011380 con 0x7f0a1009b250 2024-03-27T08:44:45.317 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.323+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a10072d80 con 0x7f09f40387a0 2024-03-27T08:44:45.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.326+0000 7f0a067fc640 1 -- 172.21.15.104:0/1903836221 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7f0a10072d80 con 0x7f09f40387a0 2024-03-27T08:44:45.322 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:44:45.322 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f09f40387a0 msgr2=0x7f09f403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f09f40387a0 0x7f09f403acb0 crc :-1 s=READY pgs=3888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 msgr2=0x7f0a101c75f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a101c75f0 crc :-1 s=READY pgs=13018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 shutdown_connections 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f09f40387a0 0x7f09f403acb0 unknown :-1 s=CLOSED pgs=3888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 --2- 172.21.15.104:0/1903836221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1009b250 0x7f0a101c75f0 unknown :-1 s=CLOSED pgs=13018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 >> 172.21.15.104:0/1903836221 conn(0x7f0a100a0c40 msgr2=0x7f0a101191f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 shutdown_connections 2024-03-27T08:44:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:44:45.328+0000 7f0a1806d640 1 -- 172.21.15.104:0/1903836221 wait complete. 2024-03-27T08:44:45.334 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":1264,"stamp":"2024-03-27T08:44:44.477243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":296076,"kb_used_data":14764,"kb_used_omap":0,"kb_used_meta":281280,"kb_avail":419134324,"statfs":{"total":429496729600,"available":429193547776,"internally_reserved":0,"allocated":15118336,"data_stored":6450981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":288030720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002219"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1285,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.731461+0000","last_change":"2024-03-27T08:31:05.063650+0000","last_active":"2024-03-27T08:44:41.731461+0000","last_peered":"2024-03-27T08:44:41.731461+0000","last_clean":"2024-03-27T08:44:41.731461+0000","last_became_active":"2024-03-27T08:31:05.053480+0000","last_became_peered":"2024-03-27T08:31:05.053480+0000","last_unstale":"2024-03-27T08:44:41.731461+0000","last_undegraded":"2024-03-27T08:44:41.731461+0000","last_fullsized":"2024-03-27T08:44:41.731461+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.206120+0000","last_change":"2024-03-27T08:31:05.114356+0000","last_active":"2024-03-27T08:44:40.206120+0000","last_peered":"2024-03-27T08:44:40.206120+0000","last_clean":"2024-03-27T08:44:40.206120+0000","last_became_active":"2024-03-27T08:31:05.102560+0000","last_became_peered":"2024-03-27T08:31:05.102560+0000","last_unstale":"2024-03-27T08:44:40.206120+0000","last_undegraded":"2024-03-27T08:44:40.206120+0000","last_fullsized":"2024-03-27T08:44:40.206120+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1273,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.775379+0000","last_change":"2024-03-27T08:31:05.110354+0000","last_active":"2024-03-27T08:44:41.775379+0000","last_peered":"2024-03-27T08:44:41.775379+0000","last_clean":"2024-03-27T08:44:41.775379+0000","last_became_active":"2024-03-27T08:31:05.101799+0000","last_became_peered":"2024-03-27T08:31:05.101799+0000","last_unstale":"2024-03-27T08:44:41.775379+0000","last_undegraded":"2024-03-27T08:44:41.775379+0000","last_fullsized":"2024-03-27T08:44:41.775379+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.620325+0000","last_change":"2024-03-27T08:31:05.128758+0000","last_active":"2024-03-27T08:44:41.620325+0000","last_peered":"2024-03-27T08:44:41.620325+0000","last_clean":"2024-03-27T08:44:41.620325+0000","last_became_active":"2024-03-27T08:31:05.111183+0000","last_became_peered":"2024-03-27T08:31:05.111183+0000","last_unstale":"2024-03-27T08:44:41.620325+0000","last_undegraded":"2024-03-27T08:44:41.620325+0000","last_fullsized":"2024-03-27T08:44:41.620325+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1285,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.662398+0000","last_change":"2024-03-27T08:31:05.145624+0000","last_active":"2024-03-27T08:44:41.662398+0000","last_peered":"2024-03-27T08:44:41.662398+0000","last_clean":"2024-03-27T08:44:41.662398+0000","last_became_active":"2024-03-27T08:31:05.138042+0000","last_became_peered":"2024-03-27T08:31:05.138042+0000","last_unstale":"2024-03-27T08:44:41.662398+0000","last_undegraded":"2024-03-27T08:44:41.662398+0000","last_fullsized":"2024-03-27T08:44:41.662398+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.208768+0000","last_change":"2024-03-27T08:31:05.084677+0000","last_active":"2024-03-27T08:44:40.208768+0000","last_peered":"2024-03-27T08:44:40.208768+0000","last_clean":"2024-03-27T08:44:40.208768+0000","last_became_active":"2024-03-27T08:31:05.068818+0000","last_became_peered":"2024-03-27T08:31:05.068818+0000","last_unstale":"2024-03-27T08:44:40.208768+0000","last_undegraded":"2024-03-27T08:44:40.208768+0000","last_fullsized":"2024-03-27T08:44:40.208768+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.181038+0000","last_change":"2024-03-27T08:31:05.087048+0000","last_active":"2024-03-27T08:44:40.181038+0000","last_peered":"2024-03-27T08:44:40.181038+0000","last_clean":"2024-03-27T08:44:40.181038+0000","last_became_active":"2024-03-27T08:31:05.064801+0000","last_became_peered":"2024-03-27T08:31:05.064801+0000","last_unstale":"2024-03-27T08:44:40.181038+0000","last_undegraded":"2024-03-27T08:44:40.181038+0000","last_fullsized":"2024-03-27T08:44:40.181038+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.223406+0000","last_change":"2024-03-27T08:31:05.134669+0000","last_active":"2024-03-27T08:44:40.223406+0000","last_peered":"2024-03-27T08:44:40.223406+0000","last_clean":"2024-03-27T08:44:40.223406+0000","last_became_active":"2024-03-27T08:31:05.128790+0000","last_became_peered":"2024-03-27T08:31:05.128790+0000","last_unstale":"2024-03-27T08:44:40.223406+0000","last_undegraded":"2024-03-27T08:44:40.223406+0000","last_fullsized":"2024-03-27T08:44:40.223406+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.649850+0000","last_change":"2024-03-27T08:31:05.144335+0000","last_active":"2024-03-27T08:44:41.649850+0000","last_peered":"2024-03-27T08:44:41.649850+0000","last_clean":"2024-03-27T08:44:41.649850+0000","last_became_active":"2024-03-27T08:31:05.125515+0000","last_became_peered":"2024-03-27T08:31:05.125515+0000","last_unstale":"2024-03-27T08:44:41.649850+0000","last_undegraded":"2024-03-27T08:44:41.649850+0000","last_fullsized":"2024-03-27T08:44:41.649850+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1273,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.751167+0000","last_change":"2024-03-27T08:31:05.083033+0000","last_active":"2024-03-27T08:44:41.751167+0000","last_peered":"2024-03-27T08:44:41.751167+0000","last_clean":"2024-03-27T08:44:41.751167+0000","last_became_active":"2024-03-27T08:31:05.074459+0000","last_became_peered":"2024-03-27T08:31:05.074459+0000","last_unstale":"2024-03-27T08:44:41.751167+0000","last_undegraded":"2024-03-27T08:44:41.751167+0000","last_fullsized":"2024-03-27T08:44:41.751167+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.664267+0000","last_change":"2024-03-27T08:31:05.114956+0000","last_active":"2024-03-27T08:44:41.664267+0000","last_peered":"2024-03-27T08:44:41.664267+0000","last_clean":"2024-03-27T08:44:41.664267+0000","last_became_active":"2024-03-27T08:31:05.099066+0000","last_became_peered":"2024-03-27T08:31:05.099066+0000","last_unstale":"2024-03-27T08:44:41.664267+0000","last_undegraded":"2024-03-27T08:44:41.664267+0000","last_fullsized":"2024-03-27T08:44:41.664267+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.224284+0000","last_change":"2024-03-27T08:31:05.179980+0000","last_active":"2024-03-27T08:44:40.224284+0000","last_peered":"2024-03-27T08:44:40.224284+0000","last_clean":"2024-03-27T08:44:40.224284+0000","last_became_active":"2024-03-27T08:31:05.174215+0000","last_became_peered":"2024-03-27T08:31:05.174215+0000","last_unstale":"2024-03-27T08:44:40.224284+0000","last_undegraded":"2024-03-27T08:44:40.224284+0000","last_fullsized":"2024-03-27T08:44:40.224284+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.674696+0000","last_change":"2024-03-27T08:31:05.108652+0000","last_active":"2024-03-27T08:44:41.674696+0000","last_peered":"2024-03-27T08:44:41.674696+0000","last_clean":"2024-03-27T08:44:41.674696+0000","last_became_active":"2024-03-27T08:31:05.095788+0000","last_became_peered":"2024-03-27T08:31:05.095788+0000","last_unstale":"2024-03-27T08:44:41.674696+0000","last_undegraded":"2024-03-27T08:44:41.674696+0000","last_fullsized":"2024-03-27T08:44:41.674696+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.596260+0000","last_change":"2024-03-27T08:31:05.131740+0000","last_active":"2024-03-27T08:44:41.596260+0000","last_peered":"2024-03-27T08:44:41.596260+0000","last_clean":"2024-03-27T08:44:41.596260+0000","last_became_active":"2024-03-27T08:31:05.103731+0000","last_became_peered":"2024-03-27T08:31:05.103731+0000","last_unstale":"2024-03-27T08:44:41.596260+0000","last_undegraded":"2024-03-27T08:44:41.596260+0000","last_fullsized":"2024-03-27T08:44:41.596260+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.234611+0000","last_change":"2024-03-27T08:31:05.119311+0000","last_active":"2024-03-27T08:44:40.234611+0000","last_peered":"2024-03-27T08:44:40.234611+0000","last_clean":"2024-03-27T08:44:40.234611+0000","last_became_active":"2024-03-27T08:31:05.090562+0000","last_became_peered":"2024-03-27T08:31:05.090562+0000","last_unstale":"2024-03-27T08:44:40.234611+0000","last_undegraded":"2024-03-27T08:44:40.234611+0000","last_fullsized":"2024-03-27T08:44:40.234611+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.201373+0000","last_change":"2024-03-27T08:31:05.106623+0000","last_active":"2024-03-27T08:44:40.201373+0000","last_peered":"2024-03-27T08:44:40.201373+0000","last_clean":"2024-03-27T08:44:40.201373+0000","last_became_active":"2024-03-27T08:31:05.076510+0000","last_became_peered":"2024-03-27T08:31:05.076510+0000","last_unstale":"2024-03-27T08:44:40.201373+0000","last_undegraded":"2024-03-27T08:44:40.201373+0000","last_fullsized":"2024-03-27T08:44:40.201373+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.252005+0000","last_change":"2024-03-27T08:31:05.110196+0000","last_active":"2024-03-27T08:44:40.252005+0000","last_peered":"2024-03-27T08:44:40.252005+0000","last_clean":"2024-03-27T08:44:40.252005+0000","last_became_active":"2024-03-27T08:31:05.081587+0000","last_became_peered":"2024-03-27T08:31:05.081587+0000","last_unstale":"2024-03-27T08:44:40.252005+0000","last_undegraded":"2024-03-27T08:44:40.252005+0000","last_fullsized":"2024-03-27T08:44:40.252005+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.687646+0000","last_change":"2024-03-27T08:31:05.153706+0000","last_active":"2024-03-27T08:44:41.687646+0000","last_peered":"2024-03-27T08:44:41.687646+0000","last_clean":"2024-03-27T08:44:41.687646+0000","last_became_active":"2024-03-27T08:31:05.137462+0000","last_became_peered":"2024-03-27T08:31:05.137462+0000","last_unstale":"2024-03-27T08:44:41.687646+0000","last_undegraded":"2024-03-27T08:44:41.687646+0000","last_fullsized":"2024-03-27T08:44:41.687646+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1273,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.717234+0000","last_change":"2024-03-27T08:31:05.091071+0000","last_active":"2024-03-27T08:44:41.717234+0000","last_peered":"2024-03-27T08:44:41.717234+0000","last_clean":"2024-03-27T08:44:41.717234+0000","last_became_active":"2024-03-27T08:31:05.077928+0000","last_became_peered":"2024-03-27T08:31:05.077928+0000","last_unstale":"2024-03-27T08:44:41.717234+0000","last_undegraded":"2024-03-27T08:44:41.717234+0000","last_fullsized":"2024-03-27T08:44:41.717234+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1285,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.687932+0000","last_change":"2024-03-27T08:31:05.096352+0000","last_active":"2024-03-27T08:44:41.687932+0000","last_peered":"2024-03-27T08:44:41.687932+0000","last_clean":"2024-03-27T08:44:41.687932+0000","last_became_active":"2024-03-27T08:31:05.090906+0000","last_became_peered":"2024-03-27T08:31:05.090906+0000","last_unstale":"2024-03-27T08:44:41.687932+0000","last_undegraded":"2024-03-27T08:44:41.687932+0000","last_fullsized":"2024-03-27T08:44:41.687932+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1285,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.733024+0000","last_change":"2024-03-27T08:31:05.198445+0000","last_active":"2024-03-27T08:44:41.733024+0000","last_peered":"2024-03-27T08:44:41.733024+0000","last_clean":"2024-03-27T08:44:41.733024+0000","last_became_active":"2024-03-27T08:31:05.192295+0000","last_became_peered":"2024-03-27T08:31:05.192295+0000","last_unstale":"2024-03-27T08:44:41.733024+0000","last_undegraded":"2024-03-27T08:44:41.733024+0000","last_fullsized":"2024-03-27T08:44:41.733024+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1285,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.671225+0000","last_change":"2024-03-27T08:31:05.085798+0000","last_active":"2024-03-27T08:44:41.671225+0000","last_peered":"2024-03-27T08:44:41.671225+0000","last_clean":"2024-03-27T08:44:41.671225+0000","last_became_active":"2024-03-27T08:31:05.077394+0000","last_became_peered":"2024-03-27T08:31:05.077394+0000","last_unstale":"2024-03-27T08:44:41.671225+0000","last_undegraded":"2024-03-27T08:44:41.671225+0000","last_fullsized":"2024-03-27T08:44:41.671225+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1273,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.725292+0000","last_change":"2024-03-27T08:31:05.120108+0000","last_active":"2024-03-27T08:44:41.725292+0000","last_peered":"2024-03-27T08:44:41.725292+0000","last_clean":"2024-03-27T08:44:41.725292+0000","last_became_active":"2024-03-27T08:31:05.114332+0000","last_became_peered":"2024-03-27T08:31:05.114332+0000","last_unstale":"2024-03-27T08:44:41.725292+0000","last_undegraded":"2024-03-27T08:44:41.725292+0000","last_fullsized":"2024-03-27T08:44:41.725292+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.671975+0000","last_change":"2024-03-27T08:31:05.181415+0000","last_active":"2024-03-27T08:44:41.671975+0000","last_peered":"2024-03-27T08:44:41.671975+0000","last_clean":"2024-03-27T08:44:41.671975+0000","last_became_active":"2024-03-27T08:31:05.175030+0000","last_became_peered":"2024-03-27T08:31:05.175030+0000","last_unstale":"2024-03-27T08:44:41.671975+0000","last_undegraded":"2024-03-27T08:44:41.671975+0000","last_fullsized":"2024-03-27T08:44:41.671975+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.638227+0000","last_change":"2024-03-27T08:31:05.169539+0000","last_active":"2024-03-27T08:44:41.638227+0000","last_peered":"2024-03-27T08:44:41.638227+0000","last_clean":"2024-03-27T08:44:41.638227+0000","last_became_active":"2024-03-27T08:31:05.164396+0000","last_became_peered":"2024-03-27T08:31:05.164396+0000","last_unstale":"2024-03-27T08:44:41.638227+0000","last_undegraded":"2024-03-27T08:44:41.638227+0000","last_fullsized":"2024-03-27T08:44:41.638227+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.219803+0000","last_change":"2024-03-27T08:31:05.116401+0000","last_active":"2024-03-27T08:44:40.219803+0000","last_peered":"2024-03-27T08:44:40.219803+0000","last_clean":"2024-03-27T08:44:40.219803+0000","last_became_active":"2024-03-27T08:31:05.098437+0000","last_became_peered":"2024-03-27T08:31:05.098437+0000","last_unstale":"2024-03-27T08:44:40.219803+0000","last_undegraded":"2024-03-27T08:44:40.219803+0000","last_fullsized":"2024-03-27T08:44:40.219803+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1259,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.648650+0000","last_change":"2024-03-27T08:31:05.141787+0000","last_active":"2024-03-27T08:44:41.648650+0000","last_peered":"2024-03-27T08:44:41.648650+0000","last_clean":"2024-03-27T08:44:41.648650+0000","last_became_active":"2024-03-27T08:31:05.120236+0000","last_became_peered":"2024-03-27T08:31:05.120236+0000","last_unstale":"2024-03-27T08:44:41.648650+0000","last_undegraded":"2024-03-27T08:44:41.648650+0000","last_fullsized":"2024-03-27T08:44:41.648650+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.240385+0000","last_change":"2024-03-27T08:31:05.209271+0000","last_active":"2024-03-27T08:44:40.240385+0000","last_peered":"2024-03-27T08:44:40.240385+0000","last_clean":"2024-03-27T08:44:40.240385+0000","last_became_active":"2024-03-27T08:31:05.203943+0000","last_became_peered":"2024-03-27T08:31:05.203943+0000","last_unstale":"2024-03-27T08:44:40.240385+0000","last_undegraded":"2024-03-27T08:44:40.240385+0000","last_fullsized":"2024-03-27T08:44:40.240385+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.217535+0000","last_change":"2024-03-27T08:31:05.095455+0000","last_active":"2024-03-27T08:44:40.217535+0000","last_peered":"2024-03-27T08:44:40.217535+0000","last_clean":"2024-03-27T08:44:40.217535+0000","last_became_active":"2024-03-27T08:31:05.072116+0000","last_became_peered":"2024-03-27T08:31:05.072116+0000","last_unstale":"2024-03-27T08:44:40.217535+0000","last_undegraded":"2024-03-27T08:44:40.217535+0000","last_fullsized":"2024-03-27T08:44:40.217535+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1253,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.176942+0000","last_change":"2024-03-27T08:31:05.158859+0000","last_active":"2024-03-27T08:44:40.176942+0000","last_peered":"2024-03-27T08:44:40.176942+0000","last_clean":"2024-03-27T08:44:40.176942+0000","last_became_active":"2024-03-27T08:31:05.151468+0000","last_became_peered":"2024-03-27T08:31:05.151468+0000","last_unstale":"2024-03-27T08:44:40.176942+0000","last_undegraded":"2024-03-27T08:44:40.176942+0000","last_fullsized":"2024-03-27T08:44:40.176942+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":1298,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.691673+0000","last_change":"2024-03-27T08:31:05.069384+0000","last_active":"2024-03-27T08:44:41.691673+0000","last_peered":"2024-03-27T08:44:41.691673+0000","last_clean":"2024-03-27T08:44:41.691673+0000","last_became_active":"2024-03-27T08:31:05.058450+0000","last_became_peered":"2024-03-27T08:31:05.058450+0000","last_unstale":"2024-03-27T08:44:41.691673+0000","last_undegraded":"2024-03-27T08:44:41.691673+0000","last_fullsized":"2024-03-27T08:44:41.691673+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1273,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.800916+0000","last_change":"2024-03-27T08:31:05.085448+0000","last_active":"2024-03-27T08:44:41.800916+0000","last_peered":"2024-03-27T08:44:41.800916+0000","last_clean":"2024-03-27T08:44:41.800916+0000","last_became_active":"2024-03-27T08:31:05.070139+0000","last_became_peered":"2024-03-27T08:31:05.070139+0000","last_unstale":"2024-03-27T08:44:41.800916+0000","last_undegraded":"2024-03-27T08:44:41.800916+0000","last_fullsized":"2024-03-27T08:44:41.800916+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:31:03.871404+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1873,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.788829+0000","last_change":"2024-03-27T08:26:13.349679+0000","last_active":"2024-03-27T08:44:41.788829+0000","last_peered":"2024-03-27T08:44:41.788829+0000","last_clean":"2024-03-27T08:44:41.788829+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:44:41.788829+0000","last_undegraded":"2024-03-27T08:44:41.788829+0000","last_fullsized":"2024-03-27T08:44:41.788829+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1863,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.760636+0000","last_change":"2024-03-27T08:26:13.342923+0000","last_active":"2024-03-27T08:44:41.760636+0000","last_peered":"2024-03-27T08:44:41.760636+0000","last_clean":"2024-03-27T08:44:41.760636+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:44:41.760636+0000","last_undegraded":"2024-03-27T08:44:41.760636+0000","last_fullsized":"2024-03-27T08:44:41.760636+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1865,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.858442+0000","last_change":"2024-03-27T08:26:13.346210+0000","last_active":"2024-03-27T08:44:41.858442+0000","last_peered":"2024-03-27T08:44:41.858442+0000","last_clean":"2024-03-27T08:44:41.858442+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:44:41.858442+0000","last_undegraded":"2024-03-27T08:44:41.858442+0000","last_fullsized":"2024-03-27T08:44:41.858442+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1863,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.719155+0000","last_change":"2024-03-27T08:26:13.333247+0000","last_active":"2024-03-27T08:44:41.719155+0000","last_peered":"2024-03-27T08:44:41.719155+0000","last_clean":"2024-03-27T08:44:41.719155+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:44:41.719155+0000","last_undegraded":"2024-03-27T08:44:41.719155+0000","last_fullsized":"2024-03-27T08:44:41.719155+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1849,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.451352+0000","last_change":"2024-03-27T08:26:13.347337+0000","last_active":"2024-03-27T08:44:40.451352+0000","last_peered":"2024-03-27T08:44:40.451352+0000","last_clean":"2024-03-27T08:44:40.451352+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:44:40.451352+0000","last_undegraded":"2024-03-27T08:44:40.451352+0000","last_fullsized":"2024-03-27T08:44:40.451352+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1873,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.668958+0000","last_change":"2024-03-27T08:26:13.336420+0000","last_active":"2024-03-27T08:44:41.668958+0000","last_peered":"2024-03-27T08:44:41.668958+0000","last_clean":"2024-03-27T08:44:41.668958+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:44:41.668958+0000","last_undegraded":"2024-03-27T08:44:41.668958+0000","last_fullsized":"2024-03-27T08:44:41.668958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1849,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:40.273134+0000","last_change":"2024-03-27T08:26:13.341429+0000","last_active":"2024-03-27T08:44:40.273134+0000","last_peered":"2024-03-27T08:44:40.273134+0000","last_clean":"2024-03-27T08:44:40.273134+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:44:40.273134+0000","last_undegraded":"2024-03-27T08:44:40.273134+0000","last_fullsized":"2024-03-27T08:44:40.273134+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1873,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.747908+0000","last_change":"2024-03-27T08:26:13.347502+0000","last_active":"2024-03-27T08:44:41.747908+0000","last_peered":"2024-03-27T08:44:41.747908+0000","last_clean":"2024-03-27T08:44:41.747908+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:44:41.747908+0000","last_undegraded":"2024-03-27T08:44:41.747908+0000","last_fullsized":"2024-03-27T08:44:41.747908+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:12.271928+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1998,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:44:41.618159+0000","last_change":"2024-03-27T08:26:11.311391+0000","last_active":"2024-03-27T08:44:41.618159+0000","last_peered":"2024-03-27T08:44:41.618159+0000","last_clean":"2024-03-27T08:44:41.618159+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:44:41.618159+0000","last_undegraded":"2024-03-27T08:44:41.618159+0000","last_fullsized":"2024-03-27T08:44:41.618159+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:26:10.263131+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654706777,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":78944,"kb_used_data":3416,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778656,"statfs":{"total":107374182400,"available":107293343744,"internally_reserved":0,"allocated":3497984,"data_stored":1318961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706777,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66460,"kb_used_data":3412,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791140,"statfs":{"total":107374182400,"available":107306127360,"internally_reserved":0,"allocated":3493888,"data_stored":1314805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706777,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83664,"kb_used_data":3976,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773936,"statfs":{"total":107374182400,"available":107288510464,"internally_reserved":0,"allocated":4071424,"data_stored":1908803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654706777,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":67008,"kb_used_data":3960,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104790592,"statfs":{"total":107374182400,"available":107305566208,"internally_reserved":0,"allocated":4055040,"data_stored":1908412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:44:45.339 INFO:tasks.ceph:pgid 75.1f last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.339 INFO:tasks.ceph:pgid 75.1e last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.339 INFO:tasks.ceph:pgid 75.1d last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.339 INFO:tasks.ceph:pgid 75.1c last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.339 INFO:tasks.ceph:pgid 75.1b last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.1a last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.19 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.18 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.17 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.16 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.15 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.14 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.13 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.340 INFO:tasks.ceph:pgid 75.12 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.11 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.10 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.f last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.e last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.d last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.c last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.b last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.a last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.9 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.341 INFO:tasks.ceph:pgid 75.8 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.7 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.6 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.5 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.4 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.3 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.2 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.1 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 75.0 last_scrub_stamp 2024-03-27T08:31:03.871404+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=31, tm_sec=3, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.342 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-27T08:26:12.271928+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=12, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-27T08:26:10.263131+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=26, tm_sec=10, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=44, tm_sec=42, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:44:45.343 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-27T08:44:49.047 INFO:tasks.ceph.mon.a.smithi104.stderr:2024-03-27T08:44:49.052+0000 7fc2f5df9640 -1 log_channel(cluster) log [ERR] : Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-27T08:44:49.047 INFO:tasks.ceph.mon.a.smithi104.stderr:2024-03-27T08:44:49.052+0000 7fc2f5df9640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 41 pgs inconsistent (PG_DAMAGED) 2024-03-27T08:45:05.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:45:05.463 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.465+0000 7fc262ff1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:45:05.464 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.469+0000 7fc262ff1640 1 Processor -- start 2024-03-27T08:45:05.464 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.469+0000 7fc262ff1640 1 -- start start 2024-03-27T08:45:05.466 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.470+0000 7fc262ff1640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc25c09cc10 0x7fc25c135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:45:05.466 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.470+0000 7fc262ff1640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc25c12e2b0 con 0x7fc25c12da20 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.470+0000 7fc262ff1640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7fc25c06e4d0 con 0x7fc25c09cc10 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25bfff640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7fc25c09cc10 0x7fc25c135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc260d8d640 1 --1- >> v1:172.21.15.104:6789/0 conn(0x7fc25c12da20 0x7fc25c09c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:35368/0 (socket says 172.21.15.104:35368) 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc260d8d640 1 -- 172.21.15.104:0/2675781160 learned_addr learned my addr 172.21.15.104:0/2675781160 (peer_addr_for_me v1:172.21.15.104:0/0) 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25bfff640 1 -- 172.21.15.104:0/2675781160 >> v1:172.21.15.104:6789/0 conn(0x7fc25c12da20 legacy=0x7fc25c09c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25bfff640 1 -- 172.21.15.104:0/2675781160 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c12dfa0 con 0x7fc25c09cc10 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25b7fe640 1 -- 172.21.15.104:0/2675781160 <== mon.0 v1:172.21.15.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290544669 0 0) 0x7fc25c12e2b0 con 0x7fc25c12da20 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25bfff640 1 --2- 172.21.15.104:0/2675781160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c135cc0 crc :-1 s=READY pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bedcddde32e229 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25b7fe640 1 -- 172.21.15.104:0/2675781160 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc250009d90 con 0x7fc25c09cc10 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25b7fe640 1 -- 172.21.15.104:0/2675781160 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc25000e230 con 0x7fc25c09cc10 2024-03-27T08:45:05.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.471+0000 7fc25b7fe640 1 -- 172.21.15.104:0/2675781160 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc25000e4d0 con 0x7fc25c09cc10 2024-03-27T08:45:05.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 -- 172.21.15.104:0/2675781160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 msgr2=0x7fc25c135cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 --2- 172.21.15.104:0/2675781160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c135cc0 crc :-1 s=READY pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 -- 172.21.15.104:0/2675781160 shutdown_connections 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 --2- 172.21.15.104:0/2675781160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c135cc0 unknown :-1 s=CLOSED pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 -- 172.21.15.104:0/2675781160 >> 172.21.15.104:0/2675781160 conn(0x7fc25c09d1d0 msgr2=0x7fc25c09d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 -- 172.21.15.104:0/2675781160 shutdown_connections 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 1 -- 172.21.15.104:0/2675781160 wait complete. 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.472+0000 7fc262ff1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.474+0000 7fc262ff1640 1 Processor -- start 2024-03-27T08:45:05.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.474+0000 7fc262ff1640 1 -- start start 2024-03-27T08:45:05.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc262ff1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c11e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc262ff1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c11f730 con 0x7fc25c09cc10 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc260d8d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c11e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc260d8d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c11e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42074/0 (socket says 172.21.15.104:42074) 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc260d8d640 1 -- 172.21.15.104:0/2177525043 learned_addr learned my addr 172.21.15.104:0/2177525043 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc260d8d640 1 -- 172.21.15.104:0/2177525043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c11f340 con 0x7fc25c09cc10 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc260d8d640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c11e090 crc :-1 s=READY pgs=13021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc24c005a60 con 0x7fc25c09cc10 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc24c005bc0 con 0x7fc25c09cc10 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25c11fbb0 con 0x7fc25c09cc10 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.475+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc24c005e60 con 0x7fc25c09cc10 2024-03-27T08:45:05.471 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.476+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25c120170 con 0x7fc25c09cc10 2024-03-27T08:45:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.476+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fc24c00c840 con 0x7fc25c09cc10 2024-03-27T08:45:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.477+0000 7fc2597fa640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2480387a0 0x7fc24803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:45:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.477+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7fc24c0482d0 con 0x7fc25c09cc10 2024-03-27T08:45:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.477+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25c063450 con 0x7fc25c09cc10 2024-03-27T08:45:05.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.477+0000 7fc25bfff640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2480387a0 0x7fc24803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:45:05.473 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.477+0000 7fc25bfff640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2480387a0 0x7fc24803acb0 crc :-1 s=READY pgs=3889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:45:05.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.482+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fc24c00d510 con 0x7fc25c09cc10 2024-03-27T08:45:05.653 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.657+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 --> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc25c09aa30 con 0x7fc2480387a0 2024-03-27T08:45:05.656 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.661+0000 7fc2597fa640 1 -- 172.21.15.104:0/2177525043 <== mgr.4099 v2:172.21.15.177:6800/769853243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107671 (crc 0 0 0) 0x7fc250009ce0 con 0x7fc2480387a0 2024-03-27T08:45:05.657 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-03-27T08:45:05.658 INFO:teuthology.orchestra.run.smithi104.stderr:dumped all 2024-03-27T08:45:05.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2480387a0 msgr2=0x7fc24803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.659 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2480387a0 0x7fc24803acb0 crc :-1 s=READY pgs=3889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 msgr2=0x7fc25c11e090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c11e090 crc :-1 s=READY pgs=13021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 shutdown_connections 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7fc2480387a0 0x7fc24803acb0 unknown :-1 s=CLOSED pgs=3889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 --2- 172.21.15.104:0/2177525043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc25c09cc10 0x7fc25c11e090 unknown :-1 s=CLOSED pgs=13021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 >> 172.21.15.104:0/2177525043 conn(0x7fc25c09d1d0 msgr2=0x7fc25c09bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.663+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 shutdown_connections 2024-03-27T08:45:05.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.664+0000 7fc262ff1640 1 -- 172.21.15.104:0/2177525043 wait complete. 2024-03-27T08:45:05.669 INFO:teuthology.orchestra.run.smithi104.stdout:{"pg_ready":true,"pg_map":{"version":1274,"stamp":"2024-03-27T08:45:04.484364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":41,"num_shallow_scrub_errors":41,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":296076,"kb_used_data":14764,"kb_used_omap":0,"kb_used_meta":281280,"kb_avail":419134324,"statfs":{"total":429496729600,"available":429193547776,"internally_reserved":0,"allocated":15118336,"data_stored":6450981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":288030720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004160"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1287,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.100158+0000","last_change":"2024-03-27T08:44:44.127469+0000","last_active":"2024-03-27T08:44:44.100158+0000","last_peered":"2024-03-27T08:44:44.100158+0000","last_clean":"2024-03-27T08:44:44.100158+0000","last_became_active":"2024-03-27T08:31:05.053480+0000","last_became_peered":"2024-03-27T08:31:05.053480+0000","last_unstale":"2024-03-27T08:44:44.100158+0000","last_undegraded":"2024-03-27T08:44:44.100158+0000","last_fullsized":"2024-03-27T08:44:44.100158+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.127250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.127250+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.637992+0000","last_change":"2024-03-27T08:44:43.666534+0000","last_active":"2024-03-27T08:44:43.637992+0000","last_peered":"2024-03-27T08:44:43.637992+0000","last_clean":"2024-03-27T08:44:43.637992+0000","last_became_active":"2024-03-27T08:31:05.102560+0000","last_became_peered":"2024-03-27T08:31:05.102560+0000","last_unstale":"2024-03-27T08:44:43.637992+0000","last_undegraded":"2024-03-27T08:44:43.637992+0000","last_fullsized":"2024-03-27T08:44:43.637992+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.666314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.666314+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1275,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.562927+0000","last_change":"2024-03-27T08:44:44.591247+0000","last_active":"2024-03-27T08:44:44.562927+0000","last_peered":"2024-03-27T08:44:44.562927+0000","last_clean":"2024-03-27T08:44:44.562927+0000","last_became_active":"2024-03-27T08:31:05.101799+0000","last_became_peered":"2024-03-27T08:31:05.101799+0000","last_unstale":"2024-03-27T08:44:44.562927+0000","last_undegraded":"2024-03-27T08:44:44.562927+0000","last_fullsized":"2024-03-27T08:44:44.562927+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.591010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.591010+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.045964+0000","last_change":"2024-03-27T08:44:45.087874+0000","last_active":"2024-03-27T08:44:45.045964+0000","last_peered":"2024-03-27T08:44:45.045964+0000","last_clean":"2024-03-27T08:44:45.045964+0000","last_became_active":"2024-03-27T08:31:05.111183+0000","last_became_peered":"2024-03-27T08:31:05.111183+0000","last_unstale":"2024-03-27T08:44:45.045964+0000","last_undegraded":"2024-03-27T08:44:45.045964+0000","last_fullsized":"2024-03-27T08:44:45.045964+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.087601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.087601+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1287,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.098961+0000","last_change":"2024-03-27T08:44:44.134946+0000","last_active":"2024-03-27T08:44:44.098961+0000","last_peered":"2024-03-27T08:44:44.098961+0000","last_clean":"2024-03-27T08:44:44.098961+0000","last_became_active":"2024-03-27T08:31:05.138042+0000","last_became_peered":"2024-03-27T08:31:05.138042+0000","last_unstale":"2024-03-27T08:44:44.098961+0000","last_undegraded":"2024-03-27T08:44:44.098961+0000","last_fullsized":"2024-03-27T08:44:44.098961+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.134740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.134740+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.636097+0000","last_change":"2024-03-27T08:44:43.667486+0000","last_active":"2024-03-27T08:44:43.636097+0000","last_peered":"2024-03-27T08:44:43.636097+0000","last_clean":"2024-03-27T08:44:43.636097+0000","last_became_active":"2024-03-27T08:31:05.068818+0000","last_became_peered":"2024-03-27T08:31:05.068818+0000","last_unstale":"2024-03-27T08:44:43.636097+0000","last_undegraded":"2024-03-27T08:44:43.636097+0000","last_fullsized":"2024-03-27T08:44:43.636097+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.667270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.667270+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.633582+0000","last_change":"2024-03-27T08:44:43.670079+0000","last_active":"2024-03-27T08:44:43.633582+0000","last_peered":"2024-03-27T08:44:43.633582+0000","last_clean":"2024-03-27T08:44:43.633582+0000","last_became_active":"2024-03-27T08:31:05.064801+0000","last_became_peered":"2024-03-27T08:31:05.064801+0000","last_unstale":"2024-03-27T08:44:43.633582+0000","last_undegraded":"2024-03-27T08:44:43.633582+0000","last_fullsized":"2024-03-27T08:44:43.633582+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.669850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.669850+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.630786+0000","last_change":"2024-03-27T08:44:43.663648+0000","last_active":"2024-03-27T08:44:43.630786+0000","last_peered":"2024-03-27T08:44:43.630786+0000","last_clean":"2024-03-27T08:44:43.630786+0000","last_became_active":"2024-03-27T08:31:05.128790+0000","last_became_peered":"2024-03-27T08:31:05.128790+0000","last_unstale":"2024-03-27T08:44:43.630786+0000","last_undegraded":"2024-03-27T08:44:43.630786+0000","last_fullsized":"2024-03-27T08:44:43.630786+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.663452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.663452+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.041922+0000","last_change":"2024-03-27T08:44:45.083454+0000","last_active":"2024-03-27T08:44:45.041922+0000","last_peered":"2024-03-27T08:44:45.041922+0000","last_clean":"2024-03-27T08:44:45.041922+0000","last_became_active":"2024-03-27T08:31:05.125515+0000","last_became_peered":"2024-03-27T08:31:05.125515+0000","last_unstale":"2024-03-27T08:44:45.041922+0000","last_undegraded":"2024-03-27T08:44:45.041922+0000","last_fullsized":"2024-03-27T08:44:45.041922+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.083244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.083244+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1275,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.561019+0000","last_change":"2024-03-27T08:44:44.588993+0000","last_active":"2024-03-27T08:44:44.561019+0000","last_peered":"2024-03-27T08:44:44.561019+0000","last_clean":"2024-03-27T08:44:44.561019+0000","last_became_active":"2024-03-27T08:31:05.074459+0000","last_became_peered":"2024-03-27T08:31:05.074459+0000","last_unstale":"2024-03-27T08:44:44.561019+0000","last_undegraded":"2024-03-27T08:44:44.561019+0000","last_fullsized":"2024-03-27T08:44:44.561019+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.588748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.588748+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.039071+0000","last_change":"2024-03-27T08:44:45.085627+0000","last_active":"2024-03-27T08:44:45.039071+0000","last_peered":"2024-03-27T08:44:45.039071+0000","last_clean":"2024-03-27T08:44:45.039071+0000","last_became_active":"2024-03-27T08:31:05.099066+0000","last_became_peered":"2024-03-27T08:31:05.099066+0000","last_unstale":"2024-03-27T08:44:45.039071+0000","last_undegraded":"2024-03-27T08:44:45.039071+0000","last_fullsized":"2024-03-27T08:44:45.039071+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.085417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.085417+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.629095+0000","last_change":"2024-03-27T08:44:43.676780+0000","last_active":"2024-03-27T08:44:43.629095+0000","last_peered":"2024-03-27T08:44:43.629095+0000","last_clean":"2024-03-27T08:44:43.629095+0000","last_became_active":"2024-03-27T08:31:05.174215+0000","last_became_peered":"2024-03-27T08:31:05.174215+0000","last_unstale":"2024-03-27T08:44:43.629095+0000","last_undegraded":"2024-03-27T08:44:43.629095+0000","last_fullsized":"2024-03-27T08:44:43.629095+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.676590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.676590+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.035083+0000","last_change":"2024-03-27T08:44:45.089364+0000","last_active":"2024-03-27T08:44:45.035083+0000","last_peered":"2024-03-27T08:44:45.035083+0000","last_clean":"2024-03-27T08:44:45.035083+0000","last_became_active":"2024-03-27T08:31:05.095788+0000","last_became_peered":"2024-03-27T08:31:05.095788+0000","last_unstale":"2024-03-27T08:44:45.035083+0000","last_undegraded":"2024-03-27T08:44:45.035083+0000","last_fullsized":"2024-03-27T08:44:45.035083+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.089149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.089149+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.034372+0000","last_change":"2024-03-27T08:44:45.081605+0000","last_active":"2024-03-27T08:44:45.034372+0000","last_peered":"2024-03-27T08:44:45.034372+0000","last_clean":"2024-03-27T08:44:45.034372+0000","last_became_active":"2024-03-27T08:31:05.103731+0000","last_became_peered":"2024-03-27T08:31:05.103731+0000","last_unstale":"2024-03-27T08:44:45.034372+0000","last_undegraded":"2024-03-27T08:44:45.034372+0000","last_fullsized":"2024-03-27T08:44:45.034372+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.081385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.081385+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.626498+0000","last_change":"2024-03-27T08:44:43.675256+0000","last_active":"2024-03-27T08:44:43.626498+0000","last_peered":"2024-03-27T08:44:43.626498+0000","last_clean":"2024-03-27T08:44:43.626498+0000","last_became_active":"2024-03-27T08:31:05.090562+0000","last_became_peered":"2024-03-27T08:31:05.090562+0000","last_unstale":"2024-03-27T08:44:43.626498+0000","last_undegraded":"2024-03-27T08:44:43.626498+0000","last_fullsized":"2024-03-27T08:44:43.626498+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.675053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.675053+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.624313+0000","last_change":"2024-03-27T08:44:43.673589+0000","last_active":"2024-03-27T08:44:43.624313+0000","last_peered":"2024-03-27T08:44:43.624313+0000","last_clean":"2024-03-27T08:44:43.624313+0000","last_became_active":"2024-03-27T08:31:05.076510+0000","last_became_peered":"2024-03-27T08:31:05.076510+0000","last_unstale":"2024-03-27T08:44:43.624313+0000","last_undegraded":"2024-03-27T08:44:43.624313+0000","last_fullsized":"2024-03-27T08:44:43.624313+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.673391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.673391+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.623752+0000","last_change":"2024-03-27T08:44:43.662739+0000","last_active":"2024-03-27T08:44:43.623752+0000","last_peered":"2024-03-27T08:44:43.623752+0000","last_clean":"2024-03-27T08:44:43.623752+0000","last_became_active":"2024-03-27T08:31:05.081587+0000","last_became_peered":"2024-03-27T08:31:05.081587+0000","last_unstale":"2024-03-27T08:44:43.623752+0000","last_undegraded":"2024-03-27T08:44:43.623752+0000","last_fullsized":"2024-03-27T08:44:43.623752+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.662525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.662525+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.028067+0000","last_change":"2024-03-27T08:44:45.079453+0000","last_active":"2024-03-27T08:44:45.028067+0000","last_peered":"2024-03-27T08:44:45.028067+0000","last_clean":"2024-03-27T08:44:45.028067+0000","last_became_active":"2024-03-27T08:31:05.137462+0000","last_became_peered":"2024-03-27T08:31:05.137462+0000","last_unstale":"2024-03-27T08:44:45.028067+0000","last_undegraded":"2024-03-27T08:44:45.028067+0000","last_fullsized":"2024-03-27T08:44:45.028067+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.079226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.079226+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1275,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.559805+0000","last_change":"2024-03-27T08:44:44.586705+0000","last_active":"2024-03-27T08:44:44.559805+0000","last_peered":"2024-03-27T08:44:44.559805+0000","last_clean":"2024-03-27T08:44:44.559805+0000","last_became_active":"2024-03-27T08:31:05.077928+0000","last_became_peered":"2024-03-27T08:31:05.077928+0000","last_unstale":"2024-03-27T08:44:44.559805+0000","last_undegraded":"2024-03-27T08:44:44.559805+0000","last_fullsized":"2024-03-27T08:44:44.559805+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.586484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.586484+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1287,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.097936+0000","last_change":"2024-03-27T08:44:44.136538+0000","last_active":"2024-03-27T08:44:44.097936+0000","last_peered":"2024-03-27T08:44:44.097936+0000","last_clean":"2024-03-27T08:44:44.097936+0000","last_became_active":"2024-03-27T08:31:05.090906+0000","last_became_peered":"2024-03-27T08:31:05.090906+0000","last_unstale":"2024-03-27T08:44:44.097936+0000","last_undegraded":"2024-03-27T08:44:44.097936+0000","last_fullsized":"2024-03-27T08:44:44.097936+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.136342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.136342+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1287,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.096214+0000","last_change":"2024-03-27T08:44:44.131802+0000","last_active":"2024-03-27T08:44:44.096214+0000","last_peered":"2024-03-27T08:44:44.096214+0000","last_clean":"2024-03-27T08:44:44.096214+0000","last_became_active":"2024-03-27T08:31:05.192295+0000","last_became_peered":"2024-03-27T08:31:05.192295+0000","last_unstale":"2024-03-27T08:44:44.096214+0000","last_undegraded":"2024-03-27T08:44:44.096214+0000","last_fullsized":"2024-03-27T08:44:44.096214+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.131590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.131590+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1287,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.091238+0000","last_change":"2024-03-27T08:44:44.129195+0000","last_active":"2024-03-27T08:44:44.091238+0000","last_peered":"2024-03-27T08:44:44.091238+0000","last_clean":"2024-03-27T08:44:44.091238+0000","last_became_active":"2024-03-27T08:31:05.077394+0000","last_became_peered":"2024-03-27T08:31:05.077394+0000","last_unstale":"2024-03-27T08:44:44.091238+0000","last_undegraded":"2024-03-27T08:44:44.091238+0000","last_fullsized":"2024-03-27T08:44:44.091238+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.128978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.128978+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1275,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.558440+0000","last_change":"2024-03-27T08:44:44.592869+0000","last_active":"2024-03-27T08:44:44.558440+0000","last_peered":"2024-03-27T08:44:44.558440+0000","last_clean":"2024-03-27T08:44:44.558440+0000","last_became_active":"2024-03-27T08:31:05.114332+0000","last_became_peered":"2024-03-27T08:31:05.114332+0000","last_unstale":"2024-03-27T08:44:44.558440+0000","last_undegraded":"2024-03-27T08:44:44.558440+0000","last_fullsized":"2024-03-27T08:44:44.558440+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.592649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.592649+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.026385+0000","last_change":"2024-03-27T08:44:45.076864+0000","last_active":"2024-03-27T08:44:45.026385+0000","last_peered":"2024-03-27T08:44:45.026385+0000","last_clean":"2024-03-27T08:44:45.026385+0000","last_became_active":"2024-03-27T08:31:05.175030+0000","last_became_peered":"2024-03-27T08:31:05.175030+0000","last_unstale":"2024-03-27T08:44:45.026385+0000","last_undegraded":"2024-03-27T08:44:45.026385+0000","last_fullsized":"2024-03-27T08:44:45.026385+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.076639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.076639+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.024822+0000","last_change":"2024-03-27T08:44:45.070457+0000","last_active":"2024-03-27T08:44:45.024822+0000","last_peered":"2024-03-27T08:44:45.024822+0000","last_clean":"2024-03-27T08:44:45.024822+0000","last_became_active":"2024-03-27T08:31:05.164396+0000","last_became_peered":"2024-03-27T08:31:05.164396+0000","last_unstale":"2024-03-27T08:44:45.024822+0000","last_undegraded":"2024-03-27T08:44:45.024822+0000","last_fullsized":"2024-03-27T08:44:45.024822+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.070228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.070228+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.625403+0000","last_change":"2024-03-27T08:44:43.668557+0000","last_active":"2024-03-27T08:44:43.625403+0000","last_peered":"2024-03-27T08:44:43.625403+0000","last_clean":"2024-03-27T08:44:43.625403+0000","last_became_active":"2024-03-27T08:31:05.098437+0000","last_became_peered":"2024-03-27T08:31:05.098437+0000","last_unstale":"2024-03-27T08:44:43.625403+0000","last_undegraded":"2024-03-27T08:44:43.625403+0000","last_fullsized":"2024-03-27T08:44:43.625403+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.668350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.668350+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1261,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.023484+0000","last_change":"2024-03-27T08:44:45.073273+0000","last_active":"2024-03-27T08:44:45.023484+0000","last_peered":"2024-03-27T08:44:45.023484+0000","last_clean":"2024-03-27T08:44:45.023484+0000","last_became_active":"2024-03-27T08:31:05.120236+0000","last_became_peered":"2024-03-27T08:31:05.120236+0000","last_unstale":"2024-03-27T08:44:45.023484+0000","last_undegraded":"2024-03-27T08:44:45.023484+0000","last_fullsized":"2024-03-27T08:44:45.023484+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.073008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.073008+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.622239+0000","last_change":"2024-03-27T08:44:43.671793+0000","last_active":"2024-03-27T08:44:43.622239+0000","last_peered":"2024-03-27T08:44:43.622239+0000","last_clean":"2024-03-27T08:44:43.622239+0000","last_became_active":"2024-03-27T08:31:05.203943+0000","last_became_peered":"2024-03-27T08:31:05.203943+0000","last_unstale":"2024-03-27T08:44:43.622239+0000","last_undegraded":"2024-03-27T08:44:43.622239+0000","last_fullsized":"2024-03-27T08:44:43.622239+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.671604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.671604+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.620996+0000","last_change":"2024-03-27T08:44:43.656257+0000","last_active":"2024-03-27T08:44:43.620996+0000","last_peered":"2024-03-27T08:44:43.620996+0000","last_clean":"2024-03-27T08:44:43.620996+0000","last_became_active":"2024-03-27T08:31:05.072116+0000","last_became_peered":"2024-03-27T08:31:05.072116+0000","last_unstale":"2024-03-27T08:44:43.620996+0000","last_undegraded":"2024-03-27T08:44:43.620996+0000","last_fullsized":"2024-03-27T08:44:43.620996+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.656033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.656033+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1255,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.619407+0000","last_change":"2024-03-27T08:44:43.660197+0000","last_active":"2024-03-27T08:44:43.619407+0000","last_peered":"2024-03-27T08:44:43.619407+0000","last_clean":"2024-03-27T08:44:43.619407+0000","last_became_active":"2024-03-27T08:31:05.151468+0000","last_became_peered":"2024-03-27T08:31:05.151468+0000","last_unstale":"2024-03-27T08:44:43.619407+0000","last_undegraded":"2024-03-27T08:44:43.619407+0000","last_fullsized":"2024-03-27T08:44:43.619407+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.659973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.659973+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":1300,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.094432+0000","last_change":"2024-03-27T08:44:44.133516+0000","last_active":"2024-03-27T08:44:44.094432+0000","last_peered":"2024-03-27T08:44:44.094432+0000","last_clean":"2024-03-27T08:44:44.094432+0000","last_became_active":"2024-03-27T08:31:05.058450+0000","last_became_peered":"2024-03-27T08:31:05.058450+0000","last_unstale":"2024-03-27T08:44:44.094432+0000","last_undegraded":"2024-03-27T08:44:44.094432+0000","last_fullsized":"2024-03-27T08:44:44.094432+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'7","last_scrub_stamp":"2024-03-27T08:44:44.133291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.133291+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1275,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.557380+0000","last_change":"2024-03-27T08:44:44.584069+0000","last_active":"2024-03-27T08:44:44.557380+0000","last_peered":"2024-03-27T08:44:44.557380+0000","last_clean":"2024-03-27T08:44:44.557380+0000","last_became_active":"2024-03-27T08:31:05.070139+0000","last_became_peered":"2024-03-27T08:31:05.070139+0000","last_unstale":"2024-03-27T08:44:44.557380+0000","last_undegraded":"2024-03-27T08:44:44.557380+0000","last_fullsized":"2024-03-27T08:44:44.557380+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.583797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.583797+0000","last_clean_scrub_stamp":"2024-03-27T08:31:03.871404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1875,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.089709+0000","last_change":"2024-03-27T08:44:44.120899+0000","last_active":"2024-03-27T08:44:44.089709+0000","last_peered":"2024-03-27T08:44:44.089709+0000","last_clean":"2024-03-27T08:44:44.089709+0000","last_became_active":"2024-03-27T08:26:13.342937+0000","last_became_peered":"2024-03-27T08:26:13.342937+0000","last_unstale":"2024-03-27T08:44:44.089709+0000","last_undegraded":"2024-03-27T08:44:44.089709+0000","last_fullsized":"2024-03-27T08:44:44.089709+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.120686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.120686+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1865,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.555166+0000","last_change":"2024-03-27T08:44:44.578103+0000","last_active":"2024-03-27T08:44:44.555166+0000","last_peered":"2024-03-27T08:44:44.555166+0000","last_clean":"2024-03-27T08:44:44.555166+0000","last_became_active":"2024-03-27T08:26:13.338189+0000","last_became_peered":"2024-03-27T08:26:13.338189+0000","last_unstale":"2024-03-27T08:44:44.555166+0000","last_undegraded":"2024-03-27T08:44:44.555166+0000","last_fullsized":"2024-03-27T08:44:44.555166+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.577840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.577840+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.024727+0000","last_change":"2024-03-27T08:44:45.058786+0000","last_active":"2024-03-27T08:44:45.024727+0000","last_peered":"2024-03-27T08:44:45.024727+0000","last_clean":"2024-03-27T08:44:45.024727+0000","last_became_active":"2024-03-27T08:26:13.341058+0000","last_became_peered":"2024-03-27T08:26:13.341058+0000","last_unstale":"2024-03-27T08:44:45.024727+0000","last_undegraded":"2024-03-27T08:44:45.024727+0000","last_fullsized":"2024-03-27T08:44:45.024727+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:45.058360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.058360+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1865,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.555302+0000","last_change":"2024-03-27T08:44:44.581727+0000","last_active":"2024-03-27T08:44:44.555302+0000","last_peered":"2024-03-27T08:44:44.555302+0000","last_clean":"2024-03-27T08:44:44.555302+0000","last_became_active":"2024-03-27T08:26:13.327912+0000","last_became_peered":"2024-03-27T08:26:13.327912+0000","last_unstale":"2024-03-27T08:44:44.555302+0000","last_undegraded":"2024-03-27T08:44:44.555302+0000","last_fullsized":"2024-03-27T08:44:44.555302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.581474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.581474+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1851,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.619554+0000","last_change":"2024-03-27T08:44:43.632223+0000","last_active":"2024-03-27T08:44:43.619554+0000","last_peered":"2024-03-27T08:44:43.619554+0000","last_clean":"2024-03-27T08:44:43.619554+0000","last_became_active":"2024-03-27T08:26:13.341680+0000","last_became_peered":"2024-03-27T08:26:13.341680+0000","last_unstale":"2024-03-27T08:44:43.619554+0000","last_undegraded":"2024-03-27T08:44:43.619554+0000","last_fullsized":"2024-03-27T08:44:43.619554+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.631897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.631897+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1875,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.088058+0000","last_change":"2024-03-27T08:44:44.123313+0000","last_active":"2024-03-27T08:44:44.088058+0000","last_peered":"2024-03-27T08:44:44.088058+0000","last_clean":"2024-03-27T08:44:44.088058+0000","last_became_active":"2024-03-27T08:26:13.331982+0000","last_became_peered":"2024-03-27T08:26:13.331982+0000","last_unstale":"2024-03-27T08:44:44.088058+0000","last_undegraded":"2024-03-27T08:44:44.088058+0000","last_fullsized":"2024-03-27T08:44:44.088058+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.123093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.123093+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1851,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:43.619300+0000","last_change":"2024-03-27T08:44:43.634364+0000","last_active":"2024-03-27T08:44:43.619300+0000","last_peered":"2024-03-27T08:44:43.619300+0000","last_clean":"2024-03-27T08:44:43.619300+0000","last_became_active":"2024-03-27T08:26:13.336288+0000","last_became_peered":"2024-03-27T08:26:13.336288+0000","last_unstale":"2024-03-27T08:44:43.619300+0000","last_undegraded":"2024-03-27T08:44:43.619300+0000","last_fullsized":"2024-03-27T08:44:43.619300+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:43.634033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:43.634033+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1875,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:44.087548+0000","last_change":"2024-03-27T08:44:44.117559+0000","last_active":"2024-03-27T08:44:44.087548+0000","last_peered":"2024-03-27T08:44:44.087548+0000","last_clean":"2024-03-27T08:44:44.087548+0000","last_became_active":"2024-03-27T08:26:13.341812+0000","last_became_peered":"2024-03-27T08:26:13.341812+0000","last_unstale":"2024-03-27T08:44:44.087548+0000","last_undegraded":"2024-03-27T08:44:44.087548+0000","last_fullsized":"2024-03-27T08:44:44.087548+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:44:44.117202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:44.117202+0000","last_clean_scrub_stamp":"2024-03-27T08:26:12.271928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":2000,"reported_epoch":712,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:44:45.021318+0000","last_change":"2024-03-27T08:44:45.091689+0000","last_active":"2024-03-27T08:44:45.021318+0000","last_peered":"2024-03-27T08:44:45.021318+0000","last_clean":"2024-03-27T08:44:45.021318+0000","last_became_active":"2024-03-27T08:26:11.306500+0000","last_became_peered":"2024-03-27T08:26:11.306500+0000","last_unstale":"2024-03-27T08:44:45.021318+0000","last_undegraded":"2024-03-27T08:44:45.021318+0000","last_fullsized":"2024-03-27T08:44:45.021318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'97","last_scrub_stamp":"2024-03-27T08:44:45.091419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:44:45.091419+0000","last_clean_scrub_stamp":"2024-03-27T08:26:10.263131+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":32,"num_shallow_scrub_errors":32,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":8,"num_shallow_scrub_errors":8,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654706797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":78944,"kb_used_data":3416,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778656,"statfs":{"total":107374182400,"available":107293343744,"internally_reserved":0,"allocated":3497984,"data_stored":1318961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66460,"kb_used_data":3412,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791140,"statfs":{"total":107374182400,"available":107306127360,"internally_reserved":0,"allocated":3493888,"data_stored":1314805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83664,"kb_used_data":3976,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773936,"statfs":{"total":107374182400,"available":107288510464,"internally_reserved":0,"allocated":4071424,"data_stored":1908803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":9,"seq":38654706797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":67008,"kb_used_data":3960,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104790592,"statfs":{"total":107374182400,"available":107305566208,"internally_reserved":0,"allocated":4055040,"data_stored":1908412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:45:05.672 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-27T08:45:05.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.783+0000 7f214446d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:45:05.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.791+0000 7f214446d640 1 Processor -- start 2024-03-27T08:45:05.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.791+0000 7f214446d640 1 -- start start 2024-03-27T08:45:05.788 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f214446d640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f213c1315f0 0x7f213c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:45:05.788 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f214446d640 1 -- --> v1:172.21.15.104:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c0a2510 con 0x7f213c131f40 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f214446d640 1 -- --> v2:172.21.15.104:3300/0 -- mon_getmap magic: 0 -- 0x7f213c0a1270 con 0x7f213c1315f0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2142209640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f213c1315f0 0x7f213c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2142209640 1 --2- >> v2:172.21.15.104:3300/0 conn(0x7f213c1315f0 0x7f213c131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38416/0 (socket says 172.21.15.104:38416) 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2142209640 1 -- 172.21.15.104:0/1642119387 learned_addr learned my addr 172.21.15.104:0/1642119387 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2141a08640 1 --1- 172.21.15.104:0/1642119387 >> v1:172.21.15.104:6789/0 conn(0x7f213c131f40 0x7f213c1347e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.104:6789/0 says I am v1:172.21.15.104:55174/0 (socket says 172.21.15.104:55174) 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2142209640 1 -- 172.21.15.104:0/1642119387 >> v1:172.21.15.104:6789/0 conn(0x7f213c131f40 legacy=0x7f213c1347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2142209640 1 -- 172.21.15.104:0/1642119387 --> v2:172.21.15.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0a2c20 con 0x7f213c1315f0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.792+0000 7f2142209640 1 --2- 172.21.15.104:0/1642119387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c131a00 crc :-1 s=READY pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8f19cb0f930583 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f2141207640 1 -- 172.21.15.104:0/1642119387 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f213800e040 con 0x7f213c1315f0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f2141207640 1 -- 172.21.15.104:0/1642119387 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f21380091e0 con 0x7f213c1315f0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f2141207640 1 -- 172.21.15.104:0/1642119387 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2138007730 con 0x7f213c1315f0 2024-03-27T08:45:05.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 -- 172.21.15.104:0/1642119387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 msgr2=0x7f213c131a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 --2- 172.21.15.104:0/1642119387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c131a00 crc :-1 s=READY pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 -- 172.21.15.104:0/1642119387 shutdown_connections 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 --2- 172.21.15.104:0/1642119387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c131a00 unknown :-1 s=CLOSED pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 -- 172.21.15.104:0/1642119387 >> 172.21.15.104:0/1642119387 conn(0x7f213c0a3780 msgr2=0x7f213c0a3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 -- 172.21.15.104:0/1642119387 shutdown_connections 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.793+0000 7f214446d640 1 -- 172.21.15.104:0/1642119387 wait complete. 2024-03-27T08:45:05.790 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.794+0000 7f214446d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.795+0000 7f214446d640 1 Processor -- start 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.795+0000 7f214446d640 1 -- start start 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f214446d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c11d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f214446d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f213c120420 con 0x7f213c1315f0 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f2142209640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c11d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f2142209640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c11d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38424/0 (socket says 172.21.15.104:38424) 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f2142209640 1 -- 172.21.15.104:0/4215968947 learned_addr learned my addr 172.21.15.104:0/4215968947 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-03-27T08:45:05.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f2142209640 1 -- 172.21.15.104:0/4215968947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c120e80 con 0x7f213c1315f0 2024-03-27T08:45:05.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.796+0000 7f2142209640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c11d130 crc :-1 s=READY pgs=13024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:45:05.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.797+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f213800e040 con 0x7f213c1315f0 2024-03-27T08:45:05.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.797+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f21380094d0 con 0x7f213c1315f0 2024-03-27T08:45:05.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.797+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c0749d0 con 0x7f213c1315f0 2024-03-27T08:45:05.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.797+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2138010990 con 0x7f213c1315f0 2024-03-27T08:45:05.793 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.797+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c11eb90 con 0x7f213c1315f0 2024-03-27T08:45:05.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.798+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c11f390 con 0x7f213c1315f0 2024-03-27T08:45:05.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.798+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2138010b40 con 0x7f213c1315f0 2024-03-27T08:45:05.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.798+0000 7f2132ffd640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f21200387a0 0x7f212003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:45:05.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.798+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 5 ==== osd_map(712..712 src has 650..712) ==== 3403+0+0 (crc 0 0 0) 0x7f21380495a0 con 0x7f213c1315f0 2024-03-27T08:45:05.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.798+0000 7f2141a08640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f21200387a0 0x7f212003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:45:05.794 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.799+0000 7f2141a08640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f21200387a0 0x7f212003acb0 crc :-1 s=READY pgs=3890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:45:05.798 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.803+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f213c11f390 con 0x7f213c1315f0 2024-03-27T08:45:05.988 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.993+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f213c0a2c20 con 0x7f213c1315f0 2024-03-27T08:45:05.991 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.997+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (crc 0 0 0) 0x7f213c0a2c20 con 0x7f213c1315f0 2024-03-27T08:45:05.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.998+0000 7f2132ffd640 1 -- 172.21.15.104:0/4215968947 <== mon.0 v2:172.21.15.104:3300/0 8 ==== config(1 keys) ==== 35+0+0 (crc 0 0 0) 0x7f2138011b90 con 0x7f213c1315f0 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.999+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f21200387a0 msgr2=0x7f212003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.999+0000 7f214446d640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f21200387a0 0x7f212003acb0 crc :-1 s=READY pgs=3890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.999+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 msgr2=0x7f213c11d130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:05.999+0000 7f214446d640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c11d130 crc :-1 s=READY pgs=13024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:06.000+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 shutdown_connections 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:06.000+0000 7f214446d640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.177:6800/769853243,v1:172.21.15.177:6801/769853243] conn(0x7f21200387a0 0x7f212003acb0 unknown :-1 s=CLOSED pgs=3890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:06.000+0000 7f214446d640 1 --2- 172.21.15.104:0/4215968947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213c1315f0 0x7f213c11d130 unknown :-1 s=CLOSED pgs=13024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:06.000+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 >> 172.21.15.104:0/4215968947 conn(0x7f213c0a3780 msgr2=0x7f213c133750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:06.000+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 shutdown_connections 2024-03-27T08:45:05.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-03-27T08:45:06.000+0000 7f214446d640 1 -- 172.21.15.104:0/4215968947 wait complete. 2024-03-27T08:45:06.009 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-27T08:45:06.009 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-27T08:45:06.010 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-27T08:45:06.010 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:45:06.088 INFO:tasks.ceph.osd.0:Stopped 2024-03-27T08:45:06.088 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-27T08:45:06.088 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:45:06.172 INFO:tasks.ceph.osd.1:Stopped 2024-03-27T08:45:06.172 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-27T08:45:06.173 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:45:06.254 INFO:tasks.ceph.osd.2:Stopped 2024-03-27T08:45:06.254 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-27T08:45:06.254 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:45:06.278 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:45:06.283+0000 7f63df2d1640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T08:45:06.278 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:45:06.283+0000 7f63df2d1640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T08:45:06.278 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-03-27T08:45:06.283+0000 7f63df2d1640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T08:45:06.334 INFO:tasks.ceph.osd.3:Stopped 2024-03-27T08:45:06.335 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-27T08:45:06.335 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-27T08:45:06.335 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:45:06.402 INFO:tasks.ceph.mgr.x:Stopped 2024-03-27T08:45:06.402 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-27T08:45:06.403 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-27T08:45:06.403 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:45:06.416 INFO:tasks.ceph.mon.a:Stopped 2024-03-27T08:45:06.417 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-27T08:45:06.417 DEBUG:teuthology.orchestra.run.smithi104:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-27T08:45:06.488 INFO:teuthology.orchestra.run.smithi104.stdout:2024-03-27T08:44:49.053526+0000 mon.a (mon.0) 1351 : cluster [ERR] Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-27T08:45:06.488 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-27T08:45:06.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-27T08:45:06.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-27T08:45:06.627 INFO:teuthology.orchestra.run.smithi104.stdout:2024-03-27T08:44:49.053526+0000 mon.a (mon.0) 1351 : cluster [ERR] Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-27T08:45:06.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:45:06.628 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-27T08:45:06.834 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:45:06.835 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-27T08:45:06.919 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:45:06.919 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-27T08:45:07.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:45:07.018 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-27T08:45:07.164 INFO:tasks.ceph:Archiving mon data... 2024-03-27T08:45:07.166 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/data/mon.a.tgz 2024-03-27T08:45:07.166 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-03-27T08:45:07.181 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.7H523uffht 2024-03-27T08:45:07.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.7H523uffht 2024-03-27T08:45:07.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.7H523uffht 2024-03-27T08:45:07.388 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.7H523uffht is 275KB 2024-03-27T08:45:07.483 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.7H523uffht 2024-03-27T08:45:07.498 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-27T08:45:07.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-27T08:45:07.539 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-27T08:45:07.617 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-27T08:45:07.635 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-27T08:45:07.635 INFO:tasks.ceph:Archiving crash dumps... 2024-03-27T08:45:07.636 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/remote/smithi104/crash 2024-03-27T08:45:07.637 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-27T08:45:07.684 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/remote/smithi177/crash 2024-03-27T08:45:07.685 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-27T08:45:07.728 INFO:tasks.ceph:Compressing logs... 2024-03-27T08:45:07.728 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T08:45:07.731 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T08:45:07.754 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100712.log 2024-03-27T08:45:07.755 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32721.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30633.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30753.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.30633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30673.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.30753.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30633.log.gz 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.30673.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.30753.log.gz 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr: 84.8%/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.30673.log.gz 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30713.log 2024-03-27T08:45:07.814 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-27T08:45:07.815 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.30713.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30713.log.gz 2024-03-27T08:45:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98479.log 2024-03-27T08:45:07.819 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99706.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32721.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.100712.log.gz 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98479.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32721.log.gz 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100000.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99706.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100000.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99706.log.gz 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100042.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mon.a.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100000.log.gz 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97810.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99681.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97810.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100042.log.gz 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98070.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98218.log 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.99681.log.gz 2024-03-27T08:45:07.820 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98070.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97810.log.gzgzip 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99404.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98218.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.98479.log.gz 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.98070.log.gz 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98122.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99823.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99404.log: /var/log/ceph/ceph-client.admin.98122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99379.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99404.log.gz 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32570.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98122.log.gz 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.99823.log.gz 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100418.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.99379.log.gz 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96909.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32570.log: /var/log/ceph/ceph-client.admin.100418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-03-27T08:45:07.821 INFO:teuthology.orchestra.run.smithi104.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32570.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.100418.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100460.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97103.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34782.log: /var/log/ceph/ceph-client.admin.100460.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96909.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97758.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.100460.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97103.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.98218.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32387.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97439.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99151.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.32387.log.gz 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97518.log 2024-03-27T08:45:07.822 INFO:teuthology.orchestra.run.smithi104.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97758.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32573.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99151.log: /var/log/ceph/ceph-client.admin.97518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98069.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.97439.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100179.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99151.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32573.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32888.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97894.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.100179.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97670.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98069.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97894.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.32888.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97706.log 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97670.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.97103.log.gz 2024-03-27T08:45:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100130.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97706.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97894.log.gz 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100754.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.97670.log.gz 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98431.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.97706.log.gz 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96928.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.100130.log.gz 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98431.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.100754.log.gz 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32429.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100544.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96928.log.gz 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99937.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 86.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32429.log.gz /var/log/ceph/ceph-client.admin.99509.log 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:07.824 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100544.log: /var/log/ceph/ceph-client.admin.99937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96947.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.100544.log.gz 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99509.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99937.log.gz 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97996.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99509.log.gz 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97664.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97996.log: /var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97669.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97664.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.96947.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100292.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97996.log.gz 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 67.5% -- replaced with /var/log/ceph/ceph-client.admin.97664.log.gz 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97705.log 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 85.8%/var/log/ceph/ceph-client.admin.33404.log: -- replaced with /var/log/ceph/ceph-client.admin.100292.log.gz 2024-03-27T08:45:07.825 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97669.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99488.log 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.97705.log: -- /var/log/ceph/ceph-client.admin.34686.log 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97809.log 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99488.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97705.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.97518.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99572.log 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97809.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.98431.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98071.log 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99572.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97809.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99660.log 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98071.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99572.log.gz 2024-03-27T08:45:07.826 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98154.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.98071.log.gz 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99660.log.gz 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98034.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34648.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99593.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: 86.2%/var/log/ceph/ceph-client.admin.34648.log: -- replaced with /var/log/ceph/ceph-client.admin.98034.log.gz 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32720.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.99593.log: /var/log/ceph/ceph-client.admin.31424.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.98154.log.gz 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34648.log.gz 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97757.log 2024-03-27T08:45:07.827 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99593.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32720.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33143.log 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32720.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.31424.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.31424.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:gzip/var/log/ceph/ceph-client.admin.97757.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97668.log 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.33143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96538.log 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.33143.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97668.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.97757.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100839.log 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.97668.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34801.log 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96538.log: /var/log/ceph/ceph-client.admin.100839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32910.log 2024-03-27T08:45:07.828 INFO:teuthology.orchestra.run.smithi104.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96538.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.100839.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32295.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98981.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32910.log: /var/log/ceph/ceph-client.admin.32295.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34801.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32572.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32910.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98981.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32295.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32345.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32572.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32572.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100628.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32345.log.gz 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97893.log 2024-03-27T08:45:07.829 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.100628.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28840.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.100628.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100376.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.tmp-client.admin.28840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100879.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28840.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.97893.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.98981.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96757.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.100376.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99329.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100879.log: /var/log/ceph/ceph-client.admin.96757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100670.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.100879.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97808.log 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.99329.log.gz 2024-03-27T08:45:07.830 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100796.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100670.log: /var/log/ceph/ceph-client.admin.97808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr: 85.5% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.100670.log.gz -- replaced with /var/log/ceph/ceph-client.admin.96757.log.gz 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100021.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.97808.log.gz 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.33873.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100796.log.gz 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97707.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99847.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100021.log: /var/log/ceph/ceph-client.admin.97707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97960.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100021.log.gz 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33017.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr: 86.5% 89.8% -- replaced with /var/log/ceph/ceph-client.admin.99847.log.gz -- replaced with /var/log/ceph/ceph-client.admin.97707.log.gz 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100085.log 2024-03-27T08:45:07.831 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97811.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.33017.log: /var/log/ceph/ceph-client.admin.100085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100334.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.33017.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97960.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99916.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100085.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97756.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97811.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100334.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100063.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:gzip/var/log/ceph/ceph-client.admin.97756.log: -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99916.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32889.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33185.log 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100063.log.gz 2024-03-27T08:45:07.832 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97756.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99446.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.33185.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32889.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100586.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.33185.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100106.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99446.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99707.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100586.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34705.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.100106.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:gzip/var/log/ceph/ceph-client.admin.99707.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98769.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34820.log 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34705.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99707.log.gz 2024-03-27T08:45:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.98769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33037.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34705.log.gz 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.34820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99895.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.33037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32849.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr: 86.0%/var/log/ceph/ceph-client.admin.99895.log: -- replaced with /var/log/ceph/ceph-client.admin.33037.log.gz 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98033.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99895.log.gz 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99530.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32849.log: /var/log/ceph/ceph-client.admin.98033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32471.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99530.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32849.log.gz 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99874.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99530.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98033.log.gz 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32719.log 2024-03-27T08:45:07.834 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31160.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32471.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32719.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99874.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98662.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.98769.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32719.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99751.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.31160.log: /var/log/ceph/ceph-client.admin.98662.log: gzip -5 86.7% -- replaced with /var/log/ceph/ceph-client.admin.31160.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97645.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97807.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.99751.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99616.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97807.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.98662.log.gz 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31396.log 2024-03-27T08:45:07.835 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97052.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97807.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.99616.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.97645.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.31396.log: /var/log/ceph/ceph-client.admin.97052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99551.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.31396.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33101.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99467.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.33101.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99551.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100155.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.33101.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32239.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97052.log.gz 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99467.log: /var/log/ceph/ceph-client.admin.100155.log: gzip -5 87.9% --verbose -- /var/log/ceph/ceph-client.admin.97961.log 2024-03-27T08:45:07.836 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99467.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.100155.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32741.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32239.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97961.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr: 85.7%/var/log/ceph/ceph-client.admin.32741.log: -- replaced with /var/log/ceph/ceph-client.admin.97961.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99797.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32741.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31239.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99979.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.31239.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.99797.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99183.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.31239.log.gz 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96576.log 2024-03-27T08:45:07.837 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99979.log: /var/log/ceph/ceph-client.admin.99183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32571.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96576.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99979.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97997.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.32571.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.32571.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96966.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97997.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.96576.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.97997.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99775.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99615.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96794.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99775.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99615.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.96966.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96985.log 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99615.log.gz 2024-03-27T08:45:07.838 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98201.log 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99425.log 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96985.log: /var/log/ceph/ceph-client.admin.98201.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.98201.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96794.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96466.log 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97892.log 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96985.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.96466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99425.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99355.log 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100502.log 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.97892.log: /var/log/ceph/ceph-client.admin.99355.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.99355.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97892.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr:gzip 93.3% -- replaced with /var/log/ceph/ceph-client.admin.99183.log.gz 2024-03-27T08:45:07.839 INFO:teuthology.orchestra.run.smithi104.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99958.log 2024-03-27T08:45:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.100502.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.100502.log.gz 2024-03-27T08:45:07.840 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.99958.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99958.log.gz 2024-03-27T08:45:07.840 INFO:teuthology.orchestra.run.smithi104.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.96466.log.gz 2024-03-27T08:45:08.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.2.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-27T08:45:08.205 INFO:teuthology.orchestra.run.smithi104.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34820.log.gz 2024-03-27T08:45:09.241 INFO:teuthology.orchestra.run.smithi104.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-27T08:45:12.484 INFO:teuthology.orchestra.run.smithi177.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-27T08:45:12.641 INFO:teuthology.orchestra.run.smithi177.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-27T08:45:12.642 INFO:teuthology.orchestra.run.smithi104.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-27T08:45:12.643 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-27T08:45:12.643 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m4.853s 2024-03-27T08:45:12.643 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m9.313s 2024-03-27T08:45:12.643 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m0.427s 2024-03-27T08:45:13.028 INFO:teuthology.orchestra.run.smithi104.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-27T08:45:13.029 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-03-27T08:45:13.029 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m5.287s 2024-03-27T08:45:13.029 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m11.688s 2024-03-27T08:45:13.029 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m0.551s 2024-03-27T08:45:13.030 INFO:tasks.ceph:Archiving logs... 2024-03-27T08:45:13.030 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/remote/smithi104/log 2024-03-27T08:45:13.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T08:45:15.044 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/remote/smithi177/log 2024-03-27T08:45:15.045 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T08:45:16.477 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-27T08:45:16.486 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-27T08:45:16.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T08:45:16.489 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T08:45:16.589 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-27T08:45:16.597 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-27T08:45:16.597 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:45:16.600 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:45:16.613 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:45:16.616 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:45:16.617 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:45:16.618 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-03-27T08:45:16.618 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 +188us[ +192us] +/- 69ms 2024-03-27T08:45:16.618 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -401us[ -388us] +/- 79ms 2024-03-27T08:45:16.618 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +231us[ +244us] +/- 82ms 2024-03-27T08:45:16.618 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:45:16.621 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:45:16.621 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-03-27T08:45:16.621 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 65 +190us[ +192us] +/- 69ms 2024-03-27T08:45:16.621 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -388us[ -384us] +/- 79ms 2024-03-27T08:45:16.621 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +244us[ +244us] +/- 82ms 2024-03-27T08:45:16.621 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:45:16.622 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-27T08:45:16.630 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-27T08:45:16.631 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-27T08:45:16.639 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T08:45:16.730 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T08:45:16.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:45:16.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:45:16.831 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:45:16.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:45:16.863 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-27T08:45:16.872 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-27T08:45:16.880 INFO:teuthology.task.internal:Duration was 1632.906385 seconds 2024-03-27T08:45:16.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-27T08:45:16.889 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-27T08:45:16.889 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T08:45:16.892 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T08:45:16.934 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:45:16.947 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:45:17.270 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-27T08:45:17.270 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-03-27T08:45:17.270 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T08:45:17.290 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T08:45:17.290 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T08:45:17.311 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-27T08:45:17.312 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T08:45:17.332 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T08:45:17.380 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-27T08:45:17.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T08:45:17.400 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T08:45:17.516 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-27T08:45:17.525 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-27T08:45:17.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T08:45:17.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T08:45:17.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-27T08:45:17.590 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:45:17.595 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:45:17.618 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-03-27T08:45:17.645 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-03-27T08:45:17.671 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:45:17.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:45:17.690 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:45:17.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:45:17.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-27T08:45:17.735 INFO:teuthology.task.internal:Transferring archived files... 2024-03-27T08:45:17.736 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/remote/smithi104 2024-03-27T08:45:17.736 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T08:45:17.772 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881/remote/smithi177 2024-03-27T08:45:17.772 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T08:45:17.807 INFO:teuthology.task.internal:Removing archive directory... 2024-03-27T08:45:17.807 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T08:45:17.810 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T08:45:17.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-27T08:45:17.867 INFO:teuthology.task.internal:Not uploading archives. 2024-03-27T08:45:17.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-27T08:45:17.875 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-27T08:45:17.875 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T08:45:17.877 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T08:45:17.889 INFO:teuthology.orchestra.run.smithi104.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 08:45 /home/ubuntu/cephtest 2024-03-27T08:45:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: 265844 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 08:45 /home/ubuntu/cephtest 2024-03-27T08:45:17.912 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-27T08:45:17.921 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-27T08:45:17.940 INFO:teuthology.nuke:Checking targets against current locks 2024-03-27T08:45:17.967 DEBUG:teuthology.nuke:shortname: smithi104 2024-03-27T08:45:17.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T08:45:17.993 DEBUG:teuthology.nuke:shortname: smithi177 2024-03-27T08:45:17.993 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T08:45:17.996 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:08:02.636494', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVyO7I+u8QQoEIR1UvSZVM4yfR+p9m5dh/TewU1IX0dTmvsTWenBaVLx8NHfIV/mDD0qBDszdiLSUURv4ZfSfc='} 2024-03-27T08:45:18.019 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625881', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:08:02.638529', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+RDlkkGXbTGmZGsjfG3RBy0+cKR5y1laHPk2EWs5LpMJ+c0Do2UyQm7JiByO/HcdIeY5HRPYduZtEUnASxpn0='} 2024-03-27T08:45:18.043 INFO:teuthology.orchestra.console.smithi104:Power off 2024-03-27T08:45:18.043 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T08:45:18.068 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off 2024-03-27T08:45:18.068 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:18.084 INFO:teuthology.orchestra.console.smithi177:Power off 2024-03-27T08:45:18.084 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T08:45:18.103 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-03-27T08:45:18.106 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2024-03-27T08:45:18.106 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:18.128 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-27T08:45:22.104 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:22.130 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:22.215 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-03-27T08:45:22.239 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-27T08:45:26.216 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:26.241 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:26.327 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-03-27T08:45:26.352 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-27T08:45:30.333 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:30.353 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:45:30.444 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-03-27T08:45:30.462 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2024-03-27T08:45:30.546 INFO:teuthology.orchestra.console.smithi104:Power off completed 2024-03-27T08:45:30.563 INFO:teuthology.orchestra.console.smithi177:Power off completed 2024-03-27T08:45:30.668 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 1632.9063849449158 failure_reason: '"2024-03-27T08:44:49.053526+0000 mon.a (mon.0) 1351 : cluster [ERR] Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS)" in cluster log' flavor: crimson owner: scheduled_matan@teuthology success: false 2024-03-27T08:45:30.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T08:45:30.752 INFO:teuthology.run:FAIL