2024-03-27T08:11:59.901 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-27T08:11:59.901 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880 --owner scheduled_matan@teuthology --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --name matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi --verbose -- /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/orig.config.yaml 2024-03-27T08:11:59.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T08:12:00.012 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880 branch: wip-matanb-crimson-foreign-ptr description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7625880' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-foreign-ptr ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: eae914f097640d21427554e7f6aa6f1e9c4a7748 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 7338 sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: eae914f097640d21427554e7f6aa6f1e9c4a7748 targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxONJN7jzLFKzFOxCdVzc2pZrWcixdRHg3DKGiCS7tp2vvp/OUOaK15GFzgk/QKd5LXJ61keemHHi3aHNLkbGo= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqxg1zoRJJjLS/Y+WaGnlKi1sat+8mnVePItmpNfEkGm/mIKFGUIDBacx6oEa13Ls9V4E+gXhM81ybdFzRl5QA= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_08:05:55 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-27T08:12:00.012 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa; will attempt to use it 2024-03-27T08:12:00.013 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa/tasks 2024-03-27T08:12:00.013 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-27T08:12:00.015 INFO:teuthology.task.internal:Checking packages... 2024-03-27T08:12:00.040 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '022fc18aa97a30a2f3803bcc4f7d8f3544371a09' 2024-03-27T08:12:00.040 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-27T08:12:00.040 INFO:teuthology.packaging:ref: None 2024-03-27T08:12:00.040 INFO:teuthology.packaging:tag: None 2024-03-27T08:12:00.040 INFO:teuthology.packaging:branch: wip-matanb-crimson-foreign-ptr 2024-03-27T08:12:00.040 INFO:teuthology.packaging:sha1: 022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:12:00.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-foreign-ptr 2024-03-27T08:12:00.183 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2463.g022fc18a 2024-03-27T08:12:00.185 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-27T08:12:00.191 INFO:teuthology.task.internal:no buildpackages task found 2024-03-27T08:12:00.191 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-27T08:12:00.198 INFO:teuthology.task.internal:Saving configuration 2024-03-27T08:12:00.210 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-27T08:12:00.216 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T08:12:00.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:07:02.055197', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxONJN7jzLFKzFOxCdVzc2pZrWcixdRHg3DKGiCS7tp2vvp/OUOaK15GFzgk/QKd5LXJ61keemHHi3aHNLkbGo='} 2024-03-27T08:12:00.261 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:07:02.056697', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqxg1zoRJJjLS/Y+WaGnlKi1sat+8mnVePItmpNfEkGm/mIKFGUIDBacx6oEa13Ls9V4E+gXhM81ybdFzRl5QA='} 2024-03-27T08:12:00.261 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-27T08:12:00.267 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-03-27T08:12:00.267 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-27T08:12:00.267 INFO:teuthology.run_tasks:Running task console_log... 2024-03-27T08:12:00.343 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0ca2f76f70>, signals=[15]) 2024-03-27T08:12:00.343 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-27T08:12:00.349 INFO:teuthology.task.internal:Opening connections... 2024-03-27T08:12:00.349 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:12:00.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:12:00.443 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:12:00.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:12:00.532 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-27T08:12:00.540 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-03-27T08:12:00.557 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-03-27T08:12:00.557 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-03-27T08:12:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T08:12:00.615 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-03-27T08:12:00.615 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T08:12:00.615 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T08:12:00.615 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T08:12:00.615 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-03-27T08:12:00.638 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-03-27T08:12:00.656 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-03-27T08:12:00.657 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-03-27T08:12:00.714 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-03-27T08:12:00.714 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-03-27T08:12:00.714 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-03-27T08:12:00.714 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T08:12:00.715 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T08:12:00.715 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-03-27T08:12:00.738 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-27T08:12:00.747 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-27T08:12:00.754 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-27T08:12:00.754 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-03-27T08:12:00.757 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-03-27T08:12:00.773 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-27T08:12:00.780 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-27T08:12:00.780 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-03-27T08:12:00.813 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-03-27T08:12:00.920 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-27T08:12:00.972 INFO:teuthology.run_tasks:Running task kernel... 2024-03-27T08:12:00.988 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-27T08:12:00.988 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-27T08:12:00.988 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T08:12:00.989 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T08:12:01.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:12:01.005 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-27T08:12:01.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:12:01.006 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-03-27T08:12:01.062 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-361.el9.x86_64 2024-03-27T08:12:01.062 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.14.0-361.el9.x86_64 2024-03-27T08:12:01.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2024-03-27T08:12:01.067 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-361.el9.x86_64 2024-03-27T08:12:01.068 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-361.el9.x86_64 2024-03-27T08:12:01.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-03-27T08:12:03.103 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-27T08:12:03.220 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-27T08:12:04.470 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-27T08:12:04.837 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-27T08:12:05.488 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-27T08:12:06.395 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-03-27T08:12:07.495 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-27T08:12:08.304 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-27T08:12:11.429 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-27T08:12:12.566 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.1 MB 00:00 2024-03-27T08:12:12.896 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-03-27T08:12:14.347 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-03-27T08:12:14.631 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-27T08:12:15.840 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-27T08:12:19.506 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-27T08:12:20.915 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-27T08:12:21.613 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-27T08:12:21.613 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-27T08:12:21.711 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:12:21.719 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:12:21.719 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:12:21.720 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-03-27T08:12:21.721 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:12:21.721 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 76 M 2024-03-27T08:12:21.721 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 123 M 2024-03-27T08:12:21.721 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:12:21.865 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 751 kB/s | 66 kB 00:00 2024-03-27T08:12:22.632 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 17 MB/s | 15 MB 00:00 2024-03-27T08:12:23.006 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-27T08:12:23.006 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-27T08:12:23.091 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-27T08:12:23.102 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-27T08:12:23.103 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:12:23.103 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:12:23.103 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:12:23.103 INFO:teuthology.orchestra.run.smithi097.stdout:Install 4 Packages 2024-03-27T08:12:23.103 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:12:23.103 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 76 M 2024-03-27T08:12:23.104 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 123 M 2024-03-27T08:12:23.104 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:12:23.149 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 24 MB/s | 33 MB 00:01 2024-03-27T08:12:23.466 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 17 MB/s | 27 MB 00:01 2024-03-27T08:12:23.467 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:12:23.467 INFO:teuthology.orchestra.run.smithi190.stdout:Total 43 MB/s | 76 MB 00:01 2024-03-27T08:12:23.492 INFO:teuthology.orchestra.run.smithi097.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 252 kB/s | 66 kB 00:00 2024-03-27T08:12:24.129 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:12:24.477 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:12:24.477 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:12:24.917 INFO:teuthology.orchestra.run.smithi097.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 8.8 MB/s | 15 MB 00:01 2024-03-27T08:12:25.501 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:12:25.501 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:12:25.558 INFO:teuthology.orchestra.run.smithi097.stdout:(3/4): kernel-modules-core-5.14.0-432.el9.x86_6 13 MB/s | 27 MB 00:02 2024-03-27T08:12:27.100 INFO:teuthology.orchestra.run.smithi097.stdout:(4/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 8.6 MB/s | 33 MB 00:03 2024-03-27T08:12:27.103 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:12:27.104 INFO:teuthology.orchestra.run.smithi097.stdout:Total 19 MB/s | 76 MB 00:04 2024-03-27T08:12:27.767 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:12:27.967 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:12:28.133 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:12:28.133 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:12:28.566 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:12:28.658 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:12:29.115 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:12:29.116 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:12:31.398 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:12:31.536 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:12:31.573 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:12:32.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:12:32.250 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:12:34.971 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:12:35.103 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:12:36.278 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:12:36.310 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:12:39.771 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:12:39.812 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:12:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:12:44.545 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:10.357 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:10.358 INFO:teuthology.orchestra.run.smithi190.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T08:13:10.358 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:13:10.402 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:10.805 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:10.806 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:13:10.806 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:13:10.806 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:13:11.212 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:13:11.597 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:13:11.697 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:13:11.697 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:13:11.697 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-27T08:13:11.697 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-27T08:13:11.697 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-27T08:13:11.698 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-03-27T08:13:11.698 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-03-27T08:13:11.698 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:13:11.778 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:13:11.778 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:13:11.779 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 68.el9 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8264254 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T08:13:11.822 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-03-27T08:13:11.823 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:13:11.824 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T08:13:11.824 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T08:13:11.825 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-432.el9.x86_64 2024-03-27T08:13:11.825 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T08:13:12.450 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-03-27T08:13:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:13.773 INFO:teuthology.orchestra.run.smithi097.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T08:13:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:13:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:13.821 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T08:13:13.856 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-03-27T08:13:13.858 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-03-27T08:13:13.873 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.S5axmolhQA 2024-03-27T08:13:13.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.S5axmolhQA 2024-03-27T08:13:13.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.S5axmolhQA 2024-03-27T08:13:14.085 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.S5axmolhQA is 6KB 2024-03-27T08:13:14.134 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.S5axmolhQA 2024-03-27T08:13:14.148 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T08:13:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-27T08:13:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T08:13:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T08:13:14.219 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T08:13:14.220 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T08:13:14.220 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T08:13:14.220 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-27T08:13:14.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-27T08:13:14.358 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-03-27T08:13:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T08:13:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:13:14.587 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:13:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-27T08:13:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-27T08:13:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-27T08:13:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-27T08:13:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:13:14.588 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:13:14.886 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:13:15.002 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:13:15.003 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:13:15.003 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-27T08:13:15.003 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-27T08:13:15.003 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-27T08:13:15.003 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-27T08:13:15.003 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-03-27T08:13:15.003 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T08:13:15.082 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T08:13:15.082 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T08:13:15.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-03-27T08:13:15.116 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-03-27T08:13:15.116 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T08:13:15.117 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:13:15.118 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T08:13:15.119 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T08:13:15.119 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 5.14.0-432.el9.x86_64 2024-03-27T08:13:15.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T08:13:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-03-27T08:13:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T08:13:17.196 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-03-27T08:13:17.199 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-27T08:13:17.255 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.XaPpatXGlo 2024-03-27T08:13:17.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XaPpatXGlo 2024-03-27T08:13:17.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.XaPpatXGlo 2024-03-27T08:13:17.461 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.XaPpatXGlo is 6KB 2024-03-27T08:13:17.509 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.XaPpatXGlo 2024-03-27T08:13:17.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T08:13:17.593 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T08:13:17.593 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T08:13:17.593 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T08:13:17.594 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-27T08:13:17.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-27T08:13:17.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-03-27T08:13:44.368 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-03-27T08:13:44.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-27T08:13:44.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:13:47.749 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-27T08:13:47.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T08:13:47.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:14:02.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-03-27T08:14:06.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-27T08:14:11.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-27T08:14:11.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:14:14.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-03-27T08:14:15.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T08:14:15.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:14:18.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-27T08:14:26.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-27T08:14:26.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:14:30.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T08:14:30.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:14:58.742 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-03-27T08:14:59.268 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-27T08:14:59.268 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-432.el9.x86_64"... 2024-03-27T08:14:59.268 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-03-27T08:14:59.294 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-432.el9.x86_64 2024-03-27T08:14:59.294 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-27T08:14:59.295 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T08:14:59.295 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T08:15:00.296 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-03-27T08:15:00.297 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T08:15:00.347 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-03-27T08:15:00.542 DEBUG:teuthology.parallel:result is None 2024-03-27T08:15:02.071 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-27T08:15:02.572 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T08:15:02.572 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-432.el9.x86_64"... 2024-03-27T08:15:02.572 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-27T08:15:02.598 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-432.el9.x86_64 2024-03-27T08:15:02.598 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-27T08:15:02.599 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T08:15:02.599 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T08:15:03.599 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-27T08:15:03.600 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T08:15:03.650 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-03-27T08:15:03.691 DEBUG:teuthology.parallel:result is None 2024-03-27T08:15:03.691 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-27T08:15:03.699 INFO:teuthology.task.internal:Creating test directory... 2024-03-27T08:15:03.699 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T08:15:03.702 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T08:15:03.720 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-27T08:15:03.727 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-27T08:15:03.734 INFO:teuthology.task.internal:Creating archive directory... 2024-03-27T08:15:03.734 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T08:15:03.757 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T08:15:03.847 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-27T08:15:03.854 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-27T08:15:03.855 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T08:15:03.857 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T08:15:03.887 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:15:03.897 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:15:03.900 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:15:03.911 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T08:15:03.913 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-27T08:15:03.928 INFO:teuthology.task.internal:Configuring sudo... 2024-03-27T08:15:03.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T08:15:03.944 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T08:15:03.982 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-27T08:15:03.992 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-27T08:15:03.992 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T08:15:04.010 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T08:15:04.039 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:15:04.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:15:04.160 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:15:04.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:15:04.283 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:15:04.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T08:15:04.349 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:15:04.389 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T08:15:04.446 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:15:04.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T08:15:04.575 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:15:04.575 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T08:15:04.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-03-27T08:15:04.647 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-03-27T08:15:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:15:04.743 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:15:05.131 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-27T08:15:05.140 INFO:teuthology.task.internal:Starting timer... 2024-03-27T08:15:05.140 INFO:teuthology.run_tasks:Running task pcp... 2024-03-27T08:15:05.151 INFO:teuthology.run_tasks:Running task selinux... 2024-03-27T08:15:05.159 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-27T08:15:05.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-03-27T08:15:05.217 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-03-27T08:15:05.219 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-03-27T08:15:05.260 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-03-27T08:15:05.262 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-27T08:15:05.262 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-03-27T08:15:05.283 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-03-27T08:15:05.283 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-03-27T08:15:05.329 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-03-27T08:15:05.329 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-03-27T08:15:05.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:15:05.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:15:05.358 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:15:05.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:15:05.398 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-27T08:15:05.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-03-27T08:15:05.435 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-03-27T08:15:05.472 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-27T08:15:05.482 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-27T08:15:05.500 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-27T08:15:05.501 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-27T08:19:34.739 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-03-27T08:19:34.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T08:19:34.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:19:34.840 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-27T08:19:34.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T08:19:34.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-27T08:19:34.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T08:19:35.005 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-03-27T08:19:35.091 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-27T08:19:35.091 INFO:teuthology.run_tasks:Running task clock... 2024-03-27T08:19:35.103 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-27T08:19:35.103 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T08:19:35.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:19:35.106 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T08:19:35.107 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:19:35.136 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T08:19:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T08:19:35.163 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T08:19:35.181 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T08:19:35.182 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-03-27T08:19:35.197 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-03-27T08:19:35.212 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-03-27T08:19:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T08:19:35.224 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-03-27T08:19:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T08:19:35.241 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T08:19:35.257 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T08:19:35.279 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:19:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:19:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-27T08:19:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.282 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.283 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.311 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:19:35.314 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:19:35.314 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-03-27T08:19:35.314 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.314 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.314 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.314 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:19:35.315 INFO:teuthology.run_tasks:Running task install... 2024-03-27T08:19:35.323 DEBUG:teuthology.task.install:project ceph 2024-03-27T08:19:35.323 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'}} 2024-03-27T08:19:35.323 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'} 2024-03-27T08:19:35.323 INFO:teuthology.task.install:Using flavor: crimson 2024-03-27T08:19:35.330 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-27T08:19:35.330 INFO:teuthology.task.install:extra packages: [] 2024-03-27T08:19:35.330 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'tag': None, 'wait_for_package': False} 2024-03-27T08:19:35.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:19:35.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'tag': None, 'wait_for_package': False} 2024-03-27T08:19:35.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:19:35.632 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/ 2024-03-27T08:19:35.632 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2463.g022fc18a 2024-03-27T08:19:35.637 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/ 2024-03-27T08:19:35.637 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2463.g022fc18a 2024-03-27T08:19:35.785 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T08:19:35.785 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:19:35.786 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T08:19:35.815 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-27T08:19:35.815 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T08:19:35.865 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-foreign-ptr/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T08:19:35.865 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:19:35.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T08:19:35.883 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T08:19:35.895 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-27T08:19:35.895 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/022fc18aa97a30a2f3803bcc4f7d8f3544371a09/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T08:19:35.965 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T08:19:35.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T08:19:36.032 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-03-27T08:19:36.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-03-27T08:19:36.046 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T08:19:36.112 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-03-27T08:19:36.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-03-27T08:19:36.273 INFO:teuthology.orchestra.run.smithi190.stdout:74 files removed 2024-03-27T08:19:36.300 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw 2024-03-27T08:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-03-27T08:19:36.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw 2024-03-27T08:19:37.140 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 160 kB/s | 87 kB 00:00 2024-03-27T08:19:37.310 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 140 kB/s | 87 kB 00:00 2024-03-27T08:19:37.618 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 27 kB/s | 11 kB 00:00 2024-03-27T08:19:37.860 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-03-27T08:19:38.042 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 5.3 kB/s | 2.0 kB 00:00 2024-03-27T08:19:38.220 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 6.4 kB/s | 2.0 kB 00:00 2024-03-27T08:19:38.460 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-27T08:19:38.663 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-27T08:19:39.936 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-03-27T08:19:40.064 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-27T08:19:40.785 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-27T08:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-27T08:19:42.536 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-03-27T08:19:42.923 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-27T08:19:46.984 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-27T08:19:47.620 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.1 MB 00:01 2024-03-27T08:19:48.832 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-03-27T08:19:49.838 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-03-27T08:19:50.567 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-27T08:19:51.581 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-27T08:19:55.642 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-27T08:19:56.480 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-27T08:19:58.111 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 9.6 M 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.7 M 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options 2024-03-27T08:19:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.9 M 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 19 M 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 25 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 796 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 548 k 2024-03-27T08:19:58.119 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 45 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 134 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 159 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T08:19:58.120 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 315 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 293 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 99 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:19:58.121 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:19:58.122 INFO:teuthology.orchestra.run.smithi097.stdout:Install 30 Packages 2024-03-27T08:19:58.122 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 2 Packages 2024-03-27T08:19:58.122 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:19:58.122 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 58 M 2024-03-27T08:19:58.122 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:19:58.936 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:19:58.949 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 9.6 M 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.7 M 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.9 M 2024-03-27T08:19:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 19 M 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 25 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 796 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 548 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T08:19:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 45 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 134 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 159 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 315 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 293 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 99 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T08:19:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout:Install 30 Packages 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 2 Packages 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 58 M 2024-03-27T08:19:58.953 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:20:00.085 INFO:teuthology.orchestra.run.smithi097.stdout:(1/32): ceph-radosgw-19.0.0-2463.g022fc18a.el9. 5.6 MB/s | 9.6 MB 00:01 2024-03-27T08:20:00.202 INFO:teuthology.orchestra.run.smithi097.stdout:(2/32): ceph-selinux-19.0.0-2463.g022fc18a.el9. 216 kB/s | 25 kB 00:00 2024-03-27T08:20:00.386 INFO:teuthology.orchestra.run.smithi097.stdout:(3/32): libcephfs2-19.0.0-2463.g022fc18a.el9.x8 4.2 MB/s | 796 kB 00:00 2024-03-27T08:20:00.536 INFO:teuthology.orchestra.run.smithi097.stdout:(4/32): libradosstriper1-19.0.0-2463.g022fc18a. 3.6 MB/s | 548 kB 00:00 2024-03-27T08:20:01.244 INFO:teuthology.orchestra.run.smithi097.stdout:(5/32): librgw2-19.0.0-2463.g022fc18a.el9.x86_6 7.1 MB/s | 5.0 MB 00:00 2024-03-27T08:20:01.298 INFO:teuthology.orchestra.run.smithi190.stdout:(1/32): ceph-base-19.0.0-2463.g022fc18a.el9.x86 2.7 MB/s | 5.9 MB 00:02 2024-03-27T08:20:01.311 INFO:teuthology.orchestra.run.smithi097.stdout:(6/32): python3-ceph-argparse-19.0.0-2463.g022f 679 kB/s | 45 kB 00:00 2024-03-27T08:20:01.356 INFO:teuthology.orchestra.run.smithi190.stdout:(2/32): ceph-selinux-19.0.0-2463.g022fc18a.el9. 432 kB/s | 25 kB 00:00 2024-03-27T08:20:01.478 INFO:teuthology.orchestra.run.smithi097.stdout:(7/32): ceph-base-19.0.0-2463.g022fc18a.el9.x86 1.9 MB/s | 5.9 MB 00:03 2024-03-27T08:20:01.503 INFO:teuthology.orchestra.run.smithi097.stdout:(8/32): python3-ceph-common-19.0.0-2463.g022fc1 698 kB/s | 134 kB 00:00 2024-03-27T08:20:01.548 INFO:teuthology.orchestra.run.smithi190.stdout:(3/32): libcephfs2-19.0.0-2463.g022fc18a.el9.x8 4.1 MB/s | 796 kB 00:00 2024-03-27T08:20:01.612 INFO:teuthology.orchestra.run.smithi097.stdout:(9/32): python3-cephfs-19.0.0-2463.g022fc18a.el 1.2 MB/s | 159 kB 00:00 2024-03-27T08:20:01.637 INFO:teuthology.orchestra.run.smithi097.stdout:(10/32): python3-rados-19.0.0-2463.g022fc18a.el 2.3 MB/s | 315 kB 00:00 2024-03-27T08:20:01.773 INFO:teuthology.orchestra.run.smithi190.stdout:(4/32): libradosstriper1-19.0.0-2463.g022fc18a. 2.4 MB/s | 548 kB 00:00 2024-03-27T08:20:01.838 INFO:teuthology.orchestra.run.smithi097.stdout:(11/32): python3-rgw-19.0.0-2463.g022fc18a.el9. 496 kB/s | 99 kB 00:00 2024-03-27T08:20:01.863 INFO:teuthology.orchestra.run.smithi097.stdout:(12/32): python3-rbd-19.0.0-2463.g022fc18a.el9. 1.1 MB/s | 293 kB 00:00 2024-03-27T08:20:01.980 INFO:teuthology.orchestra.run.smithi097.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 747 kB/s | 106 kB 00:00 2024-03-27T08:20:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-03-27T08:20:02.022 INFO:teuthology.orchestra.run.smithi097.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-27T08:20:02.072 INFO:teuthology.orchestra.run.smithi097.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-27T08:20:02.097 INFO:teuthology.orchestra.run.smithi097.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-27T08:20:02.098 INFO:teuthology.orchestra.run.smithi190.stdout:(5/32): ceph-radosgw-19.0.0-2463.g022fc18a.el9. 3.2 MB/s | 9.6 MB 00:03 2024-03-27T08:20:02.123 INFO:teuthology.orchestra.run.smithi097.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 826 kB/s | 41 kB 00:00 2024-03-27T08:20:02.173 INFO:teuthology.orchestra.run.smithi097.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 719 kB/s | 36 kB 00:00 2024-03-27T08:20:02.191 INFO:teuthology.orchestra.run.smithi190.stdout:(6/32): python3-ceph-argparse-19.0.0-2463.g022f 492 kB/s | 45 kB 00:00 2024-03-27T08:20:02.266 INFO:teuthology.orchestra.run.smithi190.stdout:(7/32): python3-ceph-common-19.0.0-2463.g022fc1 1.7 MB/s | 134 kB 00:00 2024-03-27T08:20:02.357 INFO:teuthology.orchestra.run.smithi097.stdout:(20/32): ceph-common-19.0.0-2463.g022fc18a.el9. 4.7 MB/s | 19 MB 00:03 2024-03-27T08:20:02.358 INFO:teuthology.orchestra.run.smithi190.stdout:(8/32): python3-cephfs-19.0.0-2463.g022fc18a.el 1.7 MB/s | 159 kB 00:00 2024-03-27T08:20:02.382 INFO:teuthology.orchestra.run.smithi097.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 777 kB/s | 221 kB 00:00 2024-03-27T08:20:02.407 INFO:teuthology.orchestra.run.smithi097.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 203 kB/s | 47 kB 00:00 2024-03-27T08:20:02.458 INFO:teuthology.orchestra.run.smithi190.stdout:(9/32): python3-rados-19.0.0-2463.g022fc18a.el9 3.1 MB/s | 315 kB 00:00 2024-03-27T08:20:02.499 INFO:teuthology.orchestra.run.smithi097.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-03-27T08:20:02.574 INFO:teuthology.orchestra.run.smithi097.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2024-03-27T08:20:02.599 INFO:teuthology.orchestra.run.smithi097.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-03-27T08:20:02.642 INFO:teuthology.orchestra.run.smithi097.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-27T08:20:02.642 INFO:teuthology.orchestra.run.smithi190.stdout:(10/32): ceph-common-19.0.0-2463.g022fc18a.el9. 5.3 MB/s | 19 MB 00:03 2024-03-27T08:20:02.692 INFO:teuthology.orchestra.run.smithi097.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-03-27T08:20:02.709 INFO:teuthology.orchestra.run.smithi190.stdout:(11/32): librgw2-19.0.0-2463.g022fc18a.el9.x86_ 5.4 MB/s | 5.0 MB 00:00 2024-03-27T08:20:02.734 INFO:teuthology.orchestra.run.smithi190.stdout:(12/32): python3-rgw-19.0.0-2463.g022fc18a.el9. 1.1 MB/s | 99 kB 00:00 2024-03-27T08:20:02.759 INFO:teuthology.orchestra.run.smithi190.stdout:(13/32): python3-rbd-19.0.0-2463.g022fc18a.el9. 974 kB/s | 293 kB 00:00 2024-03-27T08:20:02.809 INFO:teuthology.orchestra.run.smithi097.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-27T08:20:02.835 INFO:teuthology.orchestra.run.smithi097.stdout:(29/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-03-27T08:20:02.901 INFO:teuthology.orchestra.run.smithi190.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 551 kB/s | 106 kB 00:00 2024-03-27T08:20:02.927 INFO:teuthology.orchestra.run.smithi190.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 382 kB/s | 44 kB 00:00 2024-03-27T08:20:02.952 INFO:teuthology.orchestra.run.smithi190.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 732 kB/s | 159 kB 00:00 2024-03-27T08:20:03.043 INFO:teuthology.orchestra.run.smithi097.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-03-27T08:20:03.110 INFO:teuthology.orchestra.run.smithi190.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 661 kB 00:00 2024-03-27T08:20:03.151 INFO:teuthology.orchestra.run.smithi097.stdout:(31/32): librbd1-19.0.0-2463.g022fc18a.el9.x86_ 12 MB/s | 3.7 MB 00:00 2024-03-27T08:20:03.152 INFO:teuthology.orchestra.run.smithi190.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 282 kB 00:00 2024-03-27T08:20:03.177 INFO:teuthology.orchestra.run.smithi190.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 183 kB/s | 41 kB 00:00 2024-03-27T08:20:03.220 INFO:teuthology.orchestra.run.smithi190.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 538 kB/s | 36 kB 00:00 2024-03-27T08:20:03.245 INFO:teuthology.orchestra.run.smithi190.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-03-27T08:20:03.353 INFO:teuthology.orchestra.run.smithi190.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-03-27T08:20:03.468 INFO:teuthology.orchestra.run.smithi097.stdout:(32/32): librados2-19.0.0-2463.g022fc18a.el9.x8 5.2 MB/s | 3.4 MB 00:00 2024-03-27T08:20:03.473 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:20:03.473 INFO:teuthology.orchestra.run.smithi097.stdout:Total 11 MB/s | 58 MB 00:05 2024-03-27T08:20:03.507 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:20:03.537 INFO:teuthology.orchestra.run.smithi190.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-03-27T08:20:03.545 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:20:03.545 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:20:03.662 INFO:teuthology.orchestra.run.smithi190.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-03-27T08:20:03.687 INFO:teuthology.orchestra.run.smithi190.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 661 kB/s | 309 kB 00:00 2024-03-27T08:20:03.754 INFO:teuthology.orchestra.run.smithi190.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-03-27T08:20:03.863 INFO:teuthology.orchestra.run.smithi190.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-03-27T08:20:03.921 INFO:teuthology.orchestra.run.smithi190.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-27T08:20:04.016 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:20:04.016 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:20:04.021 INFO:teuthology.orchestra.run.smithi190.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-27T08:20:04.081 INFO:teuthology.orchestra.run.smithi190.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-03-27T08:20:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:(31/32): librados2-19.0.0-2463.g022fc18a.el9.x8 8.1 MB/s | 3.4 MB 00:00 2024-03-27T08:20:04.397 INFO:teuthology.orchestra.run.smithi190.stdout:(32/32): librbd1-19.0.0-2463.g022fc18a.el9.x86_ 9.8 MB/s | 3.7 MB 00:00 2024-03-27T08:20:04.403 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:20:04.404 INFO:teuthology.orchestra.run.smithi190.stdout:Total 11 MB/s | 58 MB 00:05 2024-03-27T08:20:04.433 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:20:04.468 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:20:04.468 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:20:04.952 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:20:04.952 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:20:05.500 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:20:05.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-27T08:20:06.028 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:20:06.103 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:20:06.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:20:06.230 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:20:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:20:06.330 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-03-27T08:20:06.359 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-03-27T08:20:06.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-03-27T08:20:06.416 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:20:06.421 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:20:06.442 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:20:06.582 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:20:06.615 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:20:06.923 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-27T08:20:06.966 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:20:06.976 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-03-27T08:20:07.003 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:20:07.058 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:20:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:20:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 12/34 2024-03-27T08:20:07.152 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:20:07.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:20:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:20:07.200 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 14/34 2024-03-27T08:20:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:20:07.281 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-03-27T08:20:07.284 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-03-27T08:20:07.306 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:20:07.313 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-03-27T08:20:07.347 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-03-27T08:20:07.370 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:20:07.386 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-03-27T08:20:07.393 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:20:07.429 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-03-27T08:20:07.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-03-27T08:20:07.545 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:20:07.572 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-03-27T08:20:07.577 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-03-27T08:20:07.613 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-03-27T08:20:07.646 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-03-27T08:20:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-03-27T08:20:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-03-27T08:20:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-27T08:20:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-03-27T08:20:07.965 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:20:08.032 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:20:08.074 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 12/34 2024-03-27T08:20:08.137 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:20:08.163 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 14/34 2024-03-27T08:20:08.240 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-03-27T08:20:08.268 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-27T08:20:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:20:08.356 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 16/34 2024-03-27T08:20:08.392 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-03-27T08:20:08.417 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:20:08.484 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 27/34 2024-03-27T08:20:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-03-27T08:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 28/34 2024-03-27T08:20:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-03-27T08:20:08.567 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-03-27T08:20:08.607 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-03-27T08:20:08.636 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-03-27T08:20:08.667 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-03-27T08:20:08.707 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-27T08:20:09.244 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-27T08:20:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:20:09.353 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 26/34 2024-03-27T08:20:09.430 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 27/34 2024-03-27T08:20:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 28/34 2024-03-27T08:20:10.879 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:20:10.920 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:20:11.595 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:20:11.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:20:11.711 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:20:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T08:20:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T08:20:11.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:11.769 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:20:11.812 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:20:11.847 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:20:12.538 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 29/34 2024-03-27T08:20:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:20:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 30/34 2024-03-27T08:20:12.657 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T08:20:12.657 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T08:20:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:20:21.738 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:20:21.738 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-03-27T08:20:21.738 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-03-27T08:20:21.739 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-03-27T08:20:21.739 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-03-27T08:20:21.739 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-03-27T08:20:21.739 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-03-27T08:20:21.739 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-03-27T08:20:21.739 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:20:22.782 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:20:22.782 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:20:22.782 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T08:20:22.782 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:20:22.783 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:20:22.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:20:22.806 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:20:22.806 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-03-27T08:20:22.806 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-03-27T08:20:22.806 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-03-27T08:20:22.807 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-03-27T08:20:22.807 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-03-27T08:20:22.807 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-03-27T08:20:22.807 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-03-27T08:20:22.807 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:22.871 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:20:22.872 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:20:23.842 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:20:23.866 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 32/34 2024-03-27T08:20:23.866 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:20:23.867 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T08:20:23.867 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:20:23.867 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T08:20:23.867 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:23.868 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:20:23.936 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-27T08:20:23.936 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:20:24.078 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:20:24.078 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 1/34 2024-03-27T08:20:24.078 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:20:24.078 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:20:24.078 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:20:24.078 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 5/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 6/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 7/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 8/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 9/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 10/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 12/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-27T08:20:24.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:20:24.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-27T08:20:24.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 33/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 1/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 3/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 4/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 5/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_6 6/34 2024-03-27T08:20:25.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 7/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9. 8/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x8 9/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 10/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 11/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 12/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 13/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-27T08:20:25.142 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-27T08:20:25.143 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-27T08:20:25.144 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 31/34 2024-03-27T08:20:25.144 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-27T08:20:25.144 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 33/34 2024-03-27T08:20:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:20:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:25.268 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-27T08:20:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T08:20:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:25.271 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:20:25.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-test 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.018 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T08:20:26.019 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:26.020 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:20:26.084 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:31 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:20:26.234 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 87 M 2024-03-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 174 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi097.stdout:Install 11 Packages 2024-03-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 90 M 2024-03-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 819 M 2024-03-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:20:26.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-test 2024-03-27T08:20:26.862 INFO:teuthology.orchestra.run.smithi097.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-27T08:20:26.887 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:30 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:20:26.937 INFO:teuthology.orchestra.run.smithi097.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-03-27T08:20:26.979 INFO:teuthology.orchestra.run.smithi097.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-27T08:20:27.013 INFO:teuthology.orchestra.run.smithi097.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-27T08:20:27.046 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:20:27.047 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:20:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:20:27.047 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:20:27.047 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:20:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 87 M 2024-03-27T08:20:27.047 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 174 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:20:27.048 INFO:teuthology.orchestra.run.smithi190.stdout:Install 11 Packages 2024-03-27T08:20:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:27.049 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 90 M 2024-03-27T08:20:27.049 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 819 M 2024-03-27T08:20:27.049 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:20:27.054 INFO:teuthology.orchestra.run.smithi097.stdout:(5/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.1 MB/s | 90 kB 00:00 2024-03-27T08:20:27.113 INFO:teuthology.orchestra.run.smithi097.stdout:(6/11): libcephsqlite-19.0.0-2463.g022fc18a.el9 419 kB/s | 174 kB 00:00 2024-03-27T08:20:27.189 INFO:teuthology.orchestra.run.smithi097.stdout:(7/11): jq-1.6-16.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-03-27T08:20:27.381 INFO:teuthology.orchestra.run.smithi097.stdout:(8/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 1.7 MB/s | 469 kB 00:00 2024-03-27T08:20:27.423 INFO:teuthology.orchestra.run.smithi097.stdout:(9/11): libubsan-11.4.1-3.el9.x86_64.rpm 887 kB/s | 187 kB 00:00 2024-03-27T08:20:27.471 INFO:teuthology.orchestra.run.smithi190.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-27T08:20:27.538 INFO:teuthology.orchestra.run.smithi190.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-27T08:20:27.580 INFO:teuthology.orchestra.run.smithi190.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-27T08:20:27.614 INFO:teuthology.orchestra.run.smithi190.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-27T08:20:27.647 INFO:teuthology.orchestra.run.smithi190.stdout:(5/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.7 MB/s | 90 kB 00:00 2024-03-27T08:20:27.648 INFO:teuthology.orchestra.run.smithi097.stdout:(10/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 456 kB/s | 122 kB 00:00 2024-03-27T08:20:27.672 INFO:teuthology.orchestra.run.smithi190.stdout:(6/11): libcephsqlite-19.0.0-2463.g022fc18a.el9 471 kB/s | 174 kB 00:00 2024-03-27T08:20:27.806 INFO:teuthology.orchestra.run.smithi190.stdout:(7/11): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-27T08:20:27.948 INFO:teuthology.orchestra.run.smithi190.stdout:(8/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 1.7 MB/s | 469 kB 00:00 2024-03-27T08:20:28.007 INFO:teuthology.orchestra.run.smithi190.stdout:(9/11): libubsan-11.4.1-3.el9.x86_64.rpm 934 kB/s | 187 kB 00:00 2024-03-27T08:20:28.307 INFO:teuthology.orchestra.run.smithi190.stdout:(10/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 340 kB/s | 122 kB 00:00 2024-03-27T08:20:34.306 INFO:teuthology.orchestra.run.smithi097.stdout:(11/11): ceph-test-19.0.0-2463.g022fc18a.el9.x8 11 MB/s | 87 MB 00:07 2024-03-27T08:20:34.312 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:20:34.313 INFO:teuthology.orchestra.run.smithi097.stdout:Total 11 MB/s | 90 MB 00:08 2024-03-27T08:20:34.354 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:20:34.377 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:20:34.377 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:20:34.424 INFO:teuthology.orchestra.run.smithi190.stdout:(11/11): ceph-test-19.0.0-2463.g022fc18a.el9.x8 12 MB/s | 87 MB 00:07 2024-03-27T08:20:34.426 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:20:34.426 INFO:teuthology.orchestra.run.smithi190.stdout:Total 12 MB/s | 90 MB 00:07 2024-03-27T08:20:34.467 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:20:34.491 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:20:34.491 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:20:35.005 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:20:35.127 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:20:35.128 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:20:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:20:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-03-27T08:20:36.116 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-03-27T08:20:36.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:20:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:20:36.216 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:20:36.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-03-27T08:20:36.259 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-03-27T08:20:36.291 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-03-27T08:20:36.293 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-03-27T08:20:36.316 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:20:36.338 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-03-27T08:20:36.369 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-03-27T08:20:36.393 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-03-27T08:20:36.417 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-03-27T08:20:36.425 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-03-27T08:20:36.461 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-03-27T08:20:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-03-27T08:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:20:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-03-27T08:20:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-03-27T08:20:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-03-27T08:20:36.637 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-03-27T08:20:36.663 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:20:48.017 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:20:48.088 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:20:48.231 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 10/11 2024-03-27T08:20:48.322 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:20:54.362 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:20:54.362 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 1/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-03-27T08:20:54.363 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 11/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 1/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-03-27T08:20:54.450 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-03-27T08:20:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: libubsan-11.4.1-3.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:54.782 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-16.el9.x86_64 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: libubsan-11.4.1-3.el9.x86_64 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-03-27T08:20:54.866 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T08:20:54.867 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T08:20:54.867 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T08:20:54.867 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T08:20:54.867 INFO:teuthology.orchestra.run.smithi190.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-03-27T08:20:54.867 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:54.867 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:20:55.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph 2024-03-27T08:20:55.189 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph 2024-03-27T08:20:55.642 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:00 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:20:55.751 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:59 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:20:55.791 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:20:55.793 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================= 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================= 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 6.5 k 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 2.2 M 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 1.6 M 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 249 k 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 71 M 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T08:20:55.794 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T08:20:55.795 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T08:20:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================= 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout:Install 36 Packages 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 84 M 2024-03-27T08:20:55.797 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 921 M 2024-03-27T08:20:55.798 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:20:55.903 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout:========================================================================================= 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout:========================================================================================= 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 6.5 k 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 2.2 M 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 1.6 M 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 249 k 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 5.0 M 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 71 M 2024-03-27T08:20:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T08:20:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T08:20:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout:========================================================================================= 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout:Install 36 Packages 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 84 M 2024-03-27T08:20:55.909 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 921 M 2024-03-27T08:20:55.910 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:20:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:(1/36): ceph-19.0.0-2463.g022fc18a.el9.x86_64.r 20 kB/s | 6.5 kB 00:00 2024-03-27T08:20:56.482 INFO:teuthology.orchestra.run.smithi190.stdout:(1/36): ceph-19.0.0-2463.g022fc18a.el9.x86_64.r 15 kB/s | 6.5 kB 00:00 2024-03-27T08:20:56.898 INFO:teuthology.orchestra.run.smithi097.stdout:(2/36): ceph-mgr-19.0.0-2463.g022fc18a.el9.x86_ 1.7 MB/s | 1.6 MB 00:00 2024-03-27T08:20:56.948 INFO:teuthology.orchestra.run.smithi097.stdout:(3/36): ceph-mds-19.0.0-2463.g022fc18a.el9.x86_ 2.3 MB/s | 2.2 MB 00:00 2024-03-27T08:20:57.124 INFO:teuthology.orchestra.run.smithi097.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2463.g022f 1.4 MB/s | 249 kB 00:00 2024-03-27T08:20:57.198 INFO:teuthology.orchestra.run.smithi190.stdout:(2/36): ceph-mds-19.0.0-2463.g022fc18a.el9.x86_ 1.9 MB/s | 2.2 MB 00:01 2024-03-27T08:20:57.232 INFO:teuthology.orchestra.run.smithi097.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-27T08:20:57.299 INFO:teuthology.orchestra.run.smithi097.stdout:(6/36): ceph-mon-19.0.0-2463.g022fc18a.el9.x86_ 5.1 MB/s | 5.0 MB 00:00 2024-03-27T08:20:57.324 INFO:teuthology.orchestra.run.smithi097.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-03-27T08:20:57.349 INFO:teuthology.orchestra.run.smithi097.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 635 kB/s | 32 kB 00:00 2024-03-27T08:20:57.375 INFO:teuthology.orchestra.run.smithi097.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2024-03-27T08:20:57.400 INFO:teuthology.orchestra.run.smithi097.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 770 kB/s | 39 kB 00:00 2024-03-27T08:20:57.434 INFO:teuthology.orchestra.run.smithi097.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-27T08:20:57.474 INFO:teuthology.orchestra.run.smithi190.stdout:(3/36): ceph-mgr-19.0.0-2463.g022fc18a.el9.x86_ 1.1 MB/s | 1.6 MB 00:01 2024-03-27T08:20:57.509 INFO:teuthology.orchestra.run.smithi097.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 11 MB/s | 1.1 MB 00:00 2024-03-27T08:20:57.534 INFO:teuthology.orchestra.run.smithi097.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-27T08:20:57.568 INFO:teuthology.orchestra.run.smithi097.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-03-27T08:20:57.593 INFO:teuthology.orchestra.run.smithi097.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-03-27T08:20:57.635 INFO:teuthology.orchestra.run.smithi097.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 537 kB/s | 22 kB 00:00 2024-03-27T08:20:57.707 INFO:teuthology.orchestra.run.smithi190.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2463.g022f 1.0 MB/s | 249 kB 00:00 2024-03-27T08:20:57.735 INFO:teuthology.orchestra.run.smithi097.stdout:(17/36): python3-requests-2.25.1-8.el9.noarch.r 748 kB/s | 125 kB 00:00 2024-03-27T08:20:57.844 INFO:teuthology.orchestra.run.smithi097.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 404 kB/s | 43 kB 00:00 2024-03-27T08:20:57.857 INFO:teuthology.orchestra.run.smithi190.stdout:(5/36): ceph-mon-19.0.0-2463.g022fc18a.el9.x86_ 3.7 MB/s | 5.0 MB 00:01 2024-03-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 645 kB/s | 151 kB 00:00 2024-03-27T08:20:57.883 INFO:teuthology.orchestra.run.smithi190.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-27T08:20:57.895 INFO:teuthology.orchestra.run.smithi097.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-03-27T08:20:57.900 INFO:teuthology.orchestra.run.smithi190.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-27T08:20:57.925 INFO:teuthology.orchestra.run.smithi190.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 755 kB/s | 32 kB 00:00 2024-03-27T08:20:57.937 INFO:teuthology.orchestra.run.smithi097.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 5.2 MB/s | 358 kB 00:00 2024-03-27T08:20:57.950 INFO:teuthology.orchestra.run.smithi190.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 942 kB/s | 47 kB 00:00 2024-03-27T08:20:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-03-27T08:20:57.976 INFO:teuthology.orchestra.run.smithi190.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 769 kB/s | 39 kB 00:00 2024-03-27T08:20:57.979 INFO:teuthology.orchestra.run.smithi097.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 422 kB/s | 18 kB 00:00 2024-03-27T08:20:58.001 INFO:teuthology.orchestra.run.smithi190.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-27T08:20:58.005 INFO:teuthology.orchestra.run.smithi097.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-03-27T08:20:58.030 INFO:teuthology.orchestra.run.smithi097.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-03-27T08:20:58.051 INFO:teuthology.orchestra.run.smithi190.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-27T08:20:58.055 INFO:teuthology.orchestra.run.smithi097.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-03-27T08:20:58.077 INFO:teuthology.orchestra.run.smithi190.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-27T08:20:58.080 INFO:teuthology.orchestra.run.smithi097.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-03-27T08:20:58.102 INFO:teuthology.orchestra.run.smithi190.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-27T08:20:58.106 INFO:teuthology.orchestra.run.smithi097.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-03-27T08:20:58.127 INFO:teuthology.orchestra.run.smithi190.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-27T08:20:58.131 INFO:teuthology.orchestra.run.smithi097.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-27T08:20:58.148 INFO:teuthology.orchestra.run.smithi097.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 394 kB/s | 16 kB 00:00 2024-03-27T08:20:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-03-27T08:20:58.182 INFO:teuthology.orchestra.run.smithi097.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-03-27T08:20:58.199 INFO:teuthology.orchestra.run.smithi097.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 710 kB/s | 36 kB 00:00 2024-03-27T08:20:58.232 INFO:teuthology.orchestra.run.smithi097.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-27T08:20:58.266 INFO:teuthology.orchestra.run.smithi097.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 6.2 MB/s | 427 kB 00:00 2024-03-27T08:20:58.291 INFO:teuthology.orchestra.run.smithi097.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 346 kB/s | 20 kB 00:00 2024-03-27T08:20:58.478 INFO:teuthology.orchestra.run.smithi190.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 504 kB/s | 151 kB 00:00 2024-03-27T08:20:58.528 INFO:teuthology.orchestra.run.smithi190.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 870 kB/s | 43 kB 00:00 2024-03-27T08:20:58.620 INFO:teuthology.orchestra.run.smithi190.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-03-27T08:20:58.729 INFO:teuthology.orchestra.run.smithi190.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-03-27T08:20:58.787 INFO:teuthology.orchestra.run.smithi190.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-03-27T08:20:58.846 INFO:teuthology.orchestra.run.smithi190.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-03-27T08:20:58.888 INFO:teuthology.orchestra.run.smithi190.stdout:(23/36): python3-requests-2.25.1-8.el9.noarch.r 159 kB/s | 125 kB 00:00 2024-03-27T08:20:58.913 INFO:teuthology.orchestra.run.smithi190.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-03-27T08:20:58.972 INFO:teuthology.orchestra.run.smithi190.stdout:(25/36): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2024-03-27T08:20:59.022 INFO:teuthology.orchestra.run.smithi190.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 145 kB/s | 19 kB 00:00 2024-03-27T08:20:59.047 INFO:teuthology.orchestra.run.smithi190.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 615 kB/s | 46 kB 00:00 2024-03-27T08:20:59.114 INFO:teuthology.orchestra.run.smithi190.stdout:(28/36): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-03-27T08:20:59.140 INFO:teuthology.orchestra.run.smithi190.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 673 kB/s | 79 kB 00:00 2024-03-27T08:20:59.173 INFO:teuthology.orchestra.run.smithi190.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-03-27T08:20:59.199 INFO:teuthology.orchestra.run.smithi190.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-03-27T08:20:59.232 INFO:teuthology.orchestra.run.smithi190.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 609 kB/s | 36 kB 00:00 2024-03-27T08:20:59.291 INFO:teuthology.orchestra.run.smithi190.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-27T08:20:59.316 INFO:teuthology.orchestra.run.smithi190.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-03-27T08:20:59.349 INFO:teuthology.orchestra.run.smithi190.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-03-27T08:21:03.425 INFO:teuthology.orchestra.run.smithi190.stdout:(36/36): ceph-osd-19.0.0-2463.g022fc18a.el9.x86 11 MB/s | 71 MB 00:06 2024-03-27T08:21:03.428 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:03.428 INFO:teuthology.orchestra.run.smithi190.stdout:Total 11 MB/s | 84 MB 00:07 2024-03-27T08:21:03.453 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:03.497 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:03.497 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:21:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:21:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:21:05.824 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:21:05.894 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-27T08:21:05.933 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-27T08:21:05.963 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-27T08:21:06.032 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-27T08:21:06.070 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-27T08:21:06.103 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-27T08:21:06.202 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-27T08:21:06.322 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-27T08:21:06.372 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-27T08:21:06.443 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-27T08:21:06.497 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-27T08:21:06.544 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-27T08:21:06.593 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-27T08:21:06.648 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-27T08:21:06.833 INFO:teuthology.orchestra.run.smithi097.stdout:(36/36): ceph-osd-19.0.0-2463.g022fc18a.el9.x86 7.1 MB/s | 71 MB 00:09 2024-03-27T08:21:06.837 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:06.837 INFO:teuthology.orchestra.run.smithi097.stdout:Total 7.6 MB/s | 84 MB 00:11 2024-03-27T08:21:06.839 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-27T08:21:06.862 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:21:06.891 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-27T08:21:06.906 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:21:06.906 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:21:06.921 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-27T08:21:06.955 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-27T08:21:06.986 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-27T08:21:07.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-27T08:21:07.096 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:21:07.137 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-27T08:21:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-27T08:21:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-27T08:21:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-27T08:21:07.344 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-27T08:21:07.387 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-27T08:21:07.458 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-27T08:21:07.595 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-27T08:21:07.668 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:21:07.668 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:21:07.721 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:21:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 31/36 2024-03-27T08:21:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:21:07.975 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:21:07.975 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:07.975 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T08:21:07.975 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:21:07.975 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:21:07.976 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:09.279 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:21:09.351 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-27T08:21:09.395 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-27T08:21:09.429 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-27T08:21:09.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-27T08:21:09.545 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-27T08:21:09.577 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-27T08:21:09.681 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-27T08:21:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-27T08:21:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-27T08:21:09.876 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-27T08:21:09.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-27T08:21:09.977 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-27T08:21:10.026 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-27T08:21:10.082 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-27T08:21:10.266 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-27T08:21:10.323 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-27T08:21:10.362 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-27T08:21:10.397 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-27T08:21:10.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-27T08:21:10.503 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-27T08:21:10.538 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:21:10.571 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-27T08:21:10.611 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-27T08:21:10.646 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-27T08:21:10.681 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-27T08:21:10.769 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-27T08:21:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-27T08:21:10.883 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-27T08:21:11.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-27T08:21:11.137 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:21:11.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 31/36 2024-03-27T08:21:11.350 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:21:11.375 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 32/36 2024-03-27T08:21:11.376 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:11.376 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T08:21:11.376 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:21:11.376 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T08:21:11.376 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:18.742 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:21:18.768 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:21:18.768 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:18.768 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T08:21:18.769 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:21:18.769 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:21:18.769 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:19.301 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:21:19.325 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:21:19.325 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:19.325 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T08:21:19.325 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:21:19.325 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:21:19.325 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:21:19.583 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:21:19.584 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:19.584 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T08:21:19.584 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:21:19.584 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:21:19.584 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:19.657 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:21:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:21:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 33/36 2024-03-27T08:21:22.176 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:22.176 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T08:21:22.176 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:21:22.176 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T08:21:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:22.718 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:21:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 34/36 2024-03-27T08:21:22.743 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:22.743 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T08:21:22.744 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:21:22.744 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T08:21:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:22.994 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:21:23.018 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 35/36 2024-03-27T08:21:23.019 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:23.019 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T08:21:23.019 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:21:23.019 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T08:21:23.019 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:23.085 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:21:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:21:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 1/36 2024-03-27T08:21:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 3/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 4/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 5/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 6/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-27T08:21:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-27T08:21:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-27T08:21:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:21:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-27T08:21:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-27T08:21:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-27T08:21:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-27T08:21:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-27T08:21:27.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-27T08:21:27.387 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:27.387 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T08:21:27.388 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T08:21:27.389 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:27.390 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:27.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-base 2024-03-27T08:21:28.206 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:32 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:28.303 INFO:teuthology.orchestra.run.smithi190.stdout:Package ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:21:28.355 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:28.356 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:21:28.356 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:28.416 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install cephadm 2024-03-27T08:21:28.977 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:32 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:29.124 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:29.125 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:21:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:21:29.125 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 762 k 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 762 k 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 768 k 2024-03-27T08:21:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:21:29.688 INFO:teuthology.orchestra.run.smithi190.stdout:cephadm-19.0.0-2463.g022fc18a.el9.noarch.rpm 1.3 MB/s | 762 kB 00:00 2024-03-27T08:21:29.689 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:29.689 INFO:teuthology.orchestra.run.smithi190.stdout:Total 1.3 MB/s | 762 kB 00:00 2024-03-27T08:21:29.689 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:29.694 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:29.694 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:21:29.699 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:21:29.700 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:21:29.773 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 36/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 1/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 3/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 4/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 5/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9. 6/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-27T08:21:29.901 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-27T08:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-27T08:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-27T08:21:30.193 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:30.275 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-27T08:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:30.930 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T08:21:30.931 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T08:21:30.932 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:30.933 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:30.936 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:30.936 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:30.936 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:21:30.936 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:30.936 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:30.936 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:31.164 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-immutable-object-cache 2024-03-27T08:21:31.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-base 2024-03-27T08:21:31.732 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:35 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:31.776 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:36 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:31.873 INFO:teuthology.orchestra.run.smithi097.stdout:Package ceph-base-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:21:31.885 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 157 k 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:21:31.886 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:21:31.887 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:21:31.887 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:31.887 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 157 k 2024-03-27T08:21:31.887 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 640 k 2024-03-27T08:21:31.887 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:21:31.925 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:31.926 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:21:31.926 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:31.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install cephadm 2024-03-27T08:21:32.261 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-immutable-object-cache-19.0.0-2463.g022fc1 420 kB/s | 157 kB 00:00 2024-03-27T08:21:32.262 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:32.262 INFO:teuthology.orchestra.run.smithi190.stdout:Total 418 kB/s | 157 kB 00:00 2024-03-27T08:21:32.262 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:32.272 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:32.272 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:21:32.324 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:21:32.325 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:21:32.472 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:21:32.547 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:37 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:32.554 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:21:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:21:32.578 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:32.579 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T08:21:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:32.695 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 762 k 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 762 k 2024-03-27T08:21:32.697 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 768 k 2024-03-27T08:21:32.697 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:21:33.240 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm-19.0.0-2463.g022fc18a.el9.noarch.rpm 1.4 MB/s | 762 kB 00:00 2024-03-27T08:21:33.241 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:33.241 INFO:teuthology.orchestra.run.smithi097.stdout:Total 1.4 MB/s | 762 kB 00:00 2024-03-27T08:21:33.241 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:21:33.247 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:21:33.247 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:21:33.252 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:21:33.253 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:21:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:21:33.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:21:33.571 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:33.571 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:21:33.571 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:33.571 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:33.571 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:33.737 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:33.841 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:33.870 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr 2024-03-27T08:21:34.198 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:34.441 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:38 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:34.507 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/1 2024-03-27T08:21:34.507 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:34.507 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:21:34.507 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:34.507 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:34.507 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:34.538 INFO:teuthology.orchestra.run.smithi190.stdout:Package ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:34.659 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-dashboard 2024-03-27T08:21:34.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-immutable-object-cache 2024-03-27T08:21:35.216 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:39 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:35.302 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:40 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:35.366 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:35.367 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 3.5 M 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 34 k 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 16 k 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T08:21:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout:====================================================================================== 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout:Install 11 Packages 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:35.369 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 7.6 M 2024-03-27T08:21:35.370 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 100 M 2024-03-27T08:21:35.370 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:21:35.456 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 157 k 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 157 k 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 640 k 2024-03-27T08:21:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:21:35.714 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-immutable-object-cache-19.0.0-2463.g022fc1 614 kB/s | 157 kB 00:00 2024-03-27T08:21:35.714 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:35.714 INFO:teuthology.orchestra.run.smithi097.stdout:Total 611 kB/s | 157 kB 00:00 2024-03-27T08:21:35.714 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:21:35.724 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:21:35.724 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:21:35.776 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:21:35.776 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:21:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:21:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:21:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:21:36.013 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:21:36.013 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T08:21:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:36.099 INFO:teuthology.orchestra.run.smithi190.stdout:(1/11): ceph-prometheus-alerts-19.0.0-2463.g022 77 kB/s | 16 kB 00:00 2024-03-27T08:21:36.182 INFO:teuthology.orchestra.run.smithi190.stdout:(2/11): ceph-grafana-dashboards-19.0.0-2463.g02 114 kB/s | 34 kB 00:00 2024-03-27T08:21:36.258 INFO:teuthology.orchestra.run.smithi190.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 238 kB 00:00 2024-03-27T08:21:36.417 INFO:teuthology.orchestra.run.smithi190.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 3.6 MB/s | 863 kB 00:00 2024-03-27T08:21:36.542 INFO:teuthology.orchestra.run.smithi190.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-03-27T08:21:36.575 INFO:teuthology.orchestra.run.smithi190.stdout:(6/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.7 MB/s | 548 kB 00:00 2024-03-27T08:21:36.634 INFO:teuthology.orchestra.run.smithi190.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-03-27T08:21:36.684 INFO:teuthology.orchestra.run.smithi190.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 619 kB/s | 31 kB 00:00 2024-03-27T08:21:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-03-27T08:21:36.877 INFO:teuthology.orchestra.run.smithi190.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.1 MB/s | 2.0 MB 00:00 2024-03-27T08:21:37.075 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a. 1/1 2024-03-27T08:21:37.075 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:37.075 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:21:37.075 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:21:37.075 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:37.075 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:37.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr 2024-03-27T08:21:37.368 INFO:teuthology.orchestra.run.smithi190.stdout:(11/11): ceph-mgr-dashboard-19.0.0-2463.g022fc1 2.4 MB/s | 3.5 MB 00:01 2024-03-27T08:21:37.371 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:37.371 INFO:teuthology.orchestra.run.smithi190.stdout:Total 3.8 MB/s | 7.6 MB 00:02 2024-03-27T08:21:37.389 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:37.406 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:37.406 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:21:37.583 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:21:37.583 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:21:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:42 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:38.001 INFO:teuthology.orchestra.run.smithi097.stdout:Package ceph-mgr-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:21:38.054 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:38.055 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:21:38.055 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:38.055 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:21:38.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-dashboard 2024-03-27T08:21:38.140 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-03-27T08:21:38.172 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-03-27T08:21:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-03-27T08:21:38.421 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-03-27T08:21:38.475 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-03-27T08:21:38.557 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-03-27T08:21:38.602 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-03-27T08:21:38.630 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:21:38.663 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 9/11 2024-03-27T08:21:38.686 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:43 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:38.835 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 3.5 M 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T08:21:38.836 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 34 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 16 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout:Install 11 Packages 2024-03-27T08:21:38.837 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:38.838 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 7.6 M 2024-03-27T08:21:38.838 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 100 M 2024-03-27T08:21:38.838 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi097.stdout:(1/11): ceph-prometheus-alerts-19.0.0-2463.g022 43 kB/s | 16 kB 00:00 2024-03-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi097.stdout:(2/11): ceph-grafana-dashboards-19.0.0-2463.g02 84 kB/s | 34 kB 00:00 2024-03-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 10/11 2024-03-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi097.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 950 kB/s | 238 kB 00:00 2024-03-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi097.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.2 MB/s | 548 kB 00:00 2024-03-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi097.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 335 kB/s | 19 kB 00:00 2024-03-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi097.stdout:(6/11): protobuf-compiler-3.14.0-13.el9.x86_64. 1.1 MB/s | 863 kB 00:00 2024-03-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi097.stdout:(7/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 7.9 MB/s | 2.0 MB 00:00 2024-03-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi097.stdout:(8/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-03-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi097.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 615 kB/s | 31 kB 00:00 2024-03-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi097.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-03-27T08:21:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:21:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 1/11 2024-03-27T08:21:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 2/11 2024-03-27T08:21:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 3/11 2024-03-27T08:21:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-03-27T08:21:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:(11/11): ceph-mgr-dashboard-19.0.0-2463.g022fc1 2.1 MB/s | 3.5 MB 00:01 2024-03-27T08:21:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:Total 3.5 MB/s | 7.6 MB 00:02 2024-03-27T08:21:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:21:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:21:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:21:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:21:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:21:41.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-03-27T08:21:41.325 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:41.325 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:21:41.325 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T08:21:41.325 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:41.326 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:41.571 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-27T08:21:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:21:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-03-27T08:21:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-03-27T08:21:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-03-27T08:21:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-03-27T08:21:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-03-27T08:21:42.129 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:46 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-03-27T08:21:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-03-27T08:21:42.279 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 7.4 M 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T08:21:42.280 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout:Install 12 Packages 2024-03-27T08:21:42.281 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:42.282 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 37 M 2024-03-27T08:21:42.282 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 209 M 2024-03-27T08:21:42.282 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:21:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:21:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 9/11 2024-03-27T08:21:42.751 INFO:teuthology.orchestra.run.smithi190.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 197 kB/s | 32 kB 00:00 2024-03-27T08:21:42.785 INFO:teuthology.orchestra.run.smithi190.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 529 kB/s | 17 kB 00:00 2024-03-27T08:21:42.826 INFO:teuthology.orchestra.run.smithi190.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 830 kB/s | 34 kB 00:00 2024-03-27T08:21:42.902 INFO:teuthology.orchestra.run.smithi190.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.6 MB/s | 3.0 MB 00:00 2024-03-27T08:21:43.144 INFO:teuthology.orchestra.run.smithi190.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2024-03-27T08:21:43.219 INFO:teuthology.orchestra.run.smithi190.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 4.8 MB 00:00 2024-03-27T08:21:43.244 INFO:teuthology.orchestra.run.smithi190.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-03-27T08:21:43.360 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 10/11 2024-03-27T08:21:43.399 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:21:43.603 INFO:teuthology.orchestra.run.smithi190.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-27T08:21:44.470 INFO:teuthology.orchestra.run.smithi190.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 225 kB/s | 195 kB 00:00 2024-03-27T08:21:44.562 INFO:teuthology.orchestra.run.smithi190.stdout:(10/12): ceph-mgr-diskprediction-local-19.0.0-2 3.8 MB/s | 7.4 MB 00:01 2024-03-27T08:21:44.704 INFO:teuthology.orchestra.run.smithi190.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 11/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el 1/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noa 2/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9 3/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-03-27T08:21:44.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-03-27T08:21:44.713 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-03-27T08:21:45.058 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T08:21:45.059 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T08:21:45.060 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T08:21:45.060 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T08:21:45.060 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T08:21:45.060 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:45.060 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:45.129 INFO:teuthology.orchestra.run.smithi190.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 427 kB/s | 803 kB 00:01 2024-03-27T08:21:45.132 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:45.132 INFO:teuthology.orchestra.run.smithi190.stdout:Total 13 MB/s | 37 MB 00:02 2024-03-27T08:21:45.172 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:45.196 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:45.197 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:21:45.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-27T08:21:45.553 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:21:45.554 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:21:45.879 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:50 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:46.029 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:46.030 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-27T08:21:46.030 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:21:46.030 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-27T08:21:46.030 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:21:46.030 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 7.4 M 2024-03-27T08:21:46.030 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:46.031 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:21:46.032 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-27T08:21:46.032 INFO:teuthology.orchestra.run.smithi097.stdout:Install 12 Packages 2024-03-27T08:21:46.032 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:46.032 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 37 M 2024-03-27T08:21:46.032 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 209 M 2024-03-27T08:21:46.032 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:21:46.262 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:21:46.327 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-27T08:21:46.358 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-27T08:21:46.450 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:21:46.503 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-27T08:21:46.504 INFO:teuthology.orchestra.run.smithi097.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 139 kB/s | 32 kB 00:00 2024-03-27T08:21:46.587 INFO:teuthology.orchestra.run.smithi097.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 206 kB/s | 17 kB 00:00 2024-03-27T08:21:46.646 INFO:teuthology.orchestra.run.smithi097.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 586 kB/s | 34 kB 00:00 2024-03-27T08:21:46.689 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:21:46.719 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:21:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-27T08:21:46.921 INFO:teuthology.orchestra.run.smithi097.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.6 MB/s | 3.0 MB 00:00 2024-03-27T08:21:47.022 INFO:teuthology.orchestra.run.smithi097.stdout:(5/12): ceph-mgr-diskprediction-local-19.0.0-24 9.9 MB/s | 7.4 MB 00:00 2024-03-27T08:21:47.272 INFO:teuthology.orchestra.run.smithi097.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 7.6 MB/s | 4.7 MB 00:00 2024-03-27T08:21:47.297 INFO:teuthology.orchestra.run.smithi097.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 795 kB/s | 217 kB 00:00 2024-03-27T08:21:47.431 INFO:teuthology.orchestra.run.smithi097.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.4 MB/s | 4.8 MB 00:00 2024-03-27T08:21:47.482 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-27T08:21:47.581 INFO:teuthology.orchestra.run.smithi097.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-27T08:21:47.605 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-27T08:21:47.623 INFO:teuthology.orchestra.run.smithi097.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-03-27T08:21:47.906 INFO:teuthology.orchestra.run.smithi097.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-03-27T08:21:48.015 INFO:teuthology.orchestra.run.smithi097.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 569 kB/s | 245 kB 00:00 2024-03-27T08:21:48.017 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:48.017 INFO:teuthology.orchestra.run.smithi097.stdout:Total 19 MB/s | 37 MB 00:01 2024-03-27T08:21:48.057 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:21:48.080 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:21:48.080 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:21:48.446 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:21:48.446 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:21:48.818 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-27T08:21:49.156 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:21:49.229 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-27T08:21:49.261 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-27T08:21:49.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:21:49.389 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-27T08:21:49.566 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:21:49.605 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:21:49.691 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-27T08:21:49.848 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-27T08:21:49.878 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:21:50.328 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-27T08:21:50.441 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-27T08:21:51.648 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-27T08:21:52.378 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:21:52.378 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 1/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-27T08:21:52.379 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-27T08:21:52.671 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-27T08:21:52.705 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T08:21:52.794 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:52.795 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:53.208 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-rook 2024-03-27T08:21:53.775 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:57 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:53.924 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================== 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================== 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 49 k 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:21:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T08:21:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================== 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout:Install 13 Packages 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 2.0 M 2024-03-27T08:21:53.927 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 26 M 2024-03-27T08:21:53.928 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:21:54.138 INFO:teuthology.orchestra.run.smithi190.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 187 kB/s | 24 kB 00:00 2024-03-27T08:21:54.154 INFO:teuthology.orchestra.run.smithi190.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 122 kB/s | 18 kB 00:00 2024-03-27T08:21:54.213 INFO:teuthology.orchestra.run.smithi190.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-27T08:21:54.238 INFO:teuthology.orchestra.run.smithi190.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-03-27T08:21:54.264 INFO:teuthology.orchestra.run.smithi190.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-27T08:21:54.289 INFO:teuthology.orchestra.run.smithi190.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 854 kB/s | 43 kB 00:00 2024-03-27T08:21:54.381 INFO:teuthology.orchestra.run.smithi190.stdout:(7/13): ceph-mgr-rook-19.0.0-2463.g022fc18a.el9 132 kB/s | 49 kB 00:00 2024-03-27T08:21:54.440 INFO:teuthology.orchestra.run.smithi190.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 100 kB/s | 15 kB 00:00 2024-03-27T08:21:54.456 INFO:teuthology.orchestra.run.smithi190.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 167 kB/s | 32 kB 00:00 2024-03-27T08:21:54.515 INFO:teuthology.orchestra.run.smithi190.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 1.5 MB/s | 203 kB 00:00 2024-03-27T08:21:54.633 INFO:teuthology.orchestra.run.smithi190.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-03-27T08:21:54.657 INFO:teuthology.orchestra.run.smithi190.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 294 kB/s | 59 kB 00:00 2024-03-27T08:21:54.683 INFO:teuthology.orchestra.run.smithi190.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 536 kB/s | 90 kB 00:00 2024-03-27T08:21:54.686 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:54.687 INFO:teuthology.orchestra.run.smithi190.stdout:Total 2.7 MB/s | 2.0 MB 00:00 2024-03-27T08:21:54.687 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:54.700 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:54.700 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:21:54.791 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:21:54.792 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:21:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:21:55.346 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 12/12 2024-03-27T08:21:55.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc 1/12 2024-03-27T08:21:55.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-27T08:21:55.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-27T08:21:55.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-27T08:21:55.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-27T08:21:55.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-27T08:21:55.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-27T08:21:55.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-27T08:21:55.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-27T08:21:55.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-27T08:21:55.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-27T08:21:55.504 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-27T08:21:55.558 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-27T08:21:55.614 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-27T08:21:55.643 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-27T08:21:55.684 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-27T08:21:55.778 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T08:21:55.816 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T08:21:55.817 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T08:21:55.817 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T08:21:55.817 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T08:21:55.817 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T08:21:55.817 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:55.817 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:21:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-27T08:21:55.936 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-27T08:21:56.261 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-rook 2024-03-27T08:21:56.660 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-27T08:21:56.700 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-27T08:21:56.734 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-27T08:21:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-27T08:21:56.804 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:21:56.828 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:01 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:21:56.980 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 49 k 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T08:21:56.981 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-27T08:21:56.982 INFO:teuthology.orchestra.run.smithi097.stdout:Install 13 Packages 2024-03-27T08:21:56.983 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:21:56.983 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 2.0 M 2024-03-27T08:21:56.983 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 26 M 2024-03-27T08:21:56.983 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:21:57.473 INFO:teuthology.orchestra.run.smithi097.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 183 kB/s | 24 kB 00:00 2024-03-27T08:21:57.498 INFO:teuthology.orchestra.run.smithi097.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 114 kB/s | 18 kB 00:00 2024-03-27T08:21:57.582 INFO:teuthology.orchestra.run.smithi097.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-27T08:21:57.616 INFO:teuthology.orchestra.run.smithi097.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 174 kB 00:00 2024-03-27T08:21:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 1/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-27T08:21:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-27T08:21:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-27T08:21:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-27T08:21:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-27T08:21:57.642 INFO:teuthology.orchestra.run.smithi097.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-03-27T08:21:57.666 INFO:teuthology.orchestra.run.smithi097.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 853 kB/s | 43 kB 00:00 2024-03-27T08:21:57.692 INFO:teuthology.orchestra.run.smithi097.stdout:(7/13): ceph-mgr-rook-19.0.0-2463.g022fc18a.el9 140 kB/s | 49 kB 00:00 2024-03-27T08:21:57.917 INFO:teuthology.orchestra.run.smithi097.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 60 kB/s | 15 kB 00:00 2024-03-27T08:21:57.942 INFO:teuthology.orchestra.run.smithi097.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 107 kB/s | 32 kB 00:00 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T08:21:57.991 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:57.992 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:21:58.159 INFO:teuthology.orchestra.run.smithi097.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 273 kB/s | 59 kB 00:00 2024-03-27T08:21:58.184 INFO:teuthology.orchestra.run.smithi097.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 412 kB/s | 203 kB 00:00 2024-03-27T08:21:58.231 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-mgr-cephadm 2024-03-27T08:21:58.334 INFO:teuthology.orchestra.run.smithi097.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 511 kB/s | 90 kB 00:00 2024-03-27T08:21:58.468 INFO:teuthology.orchestra.run.smithi097.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-03-27T08:21:58.470 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:58.470 INFO:teuthology.orchestra.run.smithi097.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-03-27T08:21:58.471 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:21:58.483 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:21:58.484 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:21:58.575 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:21:58.575 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:21:58.794 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:02 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:21:58.944 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:21:58.945 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================== 2024-03-27T08:21:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:21:58.945 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================== 2024-03-27T08:21:58.945 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:21:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 141 k 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================== 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout:Install 6 Packages 2024-03-27T08:21:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:21:58.947 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 6.8 M 2024-03-27T08:21:58.947 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 33 M 2024-03-27T08:21:58.947 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:21:59.082 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:21:59.246 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-27T08:21:59.302 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-27T08:21:59.358 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-27T08:21:59.396 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-27T08:21:59.444 INFO:teuthology.orchestra.run.smithi190.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2463.g022fc18a.e 532 kB/s | 141 kB 00:00 2024-03-27T08:21:59.445 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-27T08:21:59.536 INFO:teuthology.orchestra.run.smithi190.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-03-27T08:21:59.538 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-27T08:21:59.639 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-27T08:21:59.689 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-27T08:21:59.769 INFO:teuthology.orchestra.run.smithi190.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 247 kB/s | 58 kB 00:00 2024-03-27T08:21:59.803 INFO:teuthology.orchestra.run.smithi190.stdout:(4/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 397 kB/s | 248 kB 00:00 2024-03-27T08:21:59.903 INFO:teuthology.orchestra.run.smithi190.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 381 kB/s | 51 kB 00:00 2024-03-27T08:21:59.945 INFO:teuthology.orchestra.run.smithi190.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-03-27T08:21:59.946 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:21:59.946 INFO:teuthology.orchestra.run.smithi190.stdout:Total 6.8 MB/s | 6.8 MB 00:00 2024-03-27T08:21:59.959 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:21:59.970 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:21:59.970 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:00.059 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:00.060 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:00.352 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:00.401 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-27T08:22:00.454 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-27T08:22:00.488 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-27T08:22:00.509 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-27T08:22:00.545 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-27T08:22:00.562 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-27T08:22:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:22:01.088 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-27T08:22:01.174 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-27T08:22:01.247 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-27T08:22:01.276 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:22:01.333 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 13/13 2024-03-27T08:22:01.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 1/13 2024-03-27T08:22:01.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-27T08:22:01.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-27T08:22:01.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-27T08:22:01.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T08:22:01.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:01.709 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:01.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-cephadm 2024-03-27T08:22:02.195 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:22:02.195 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/6 2024-03-27T08:22:02.195 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-27T08:22:02.195 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-27T08:22:02.195 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-27T08:22:02.196 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:02.498 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:02.516 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:07 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-27T08:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-27T08:22:02.668 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 141 k 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout:Install 6 Packages 2024-03-27T08:22:02.669 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 6.8 M 2024-03-27T08:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 33 M 2024-03-27T08:22:02.670 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:02.762 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-fuse 2024-03-27T08:22:03.206 INFO:teuthology.orchestra.run.smithi097.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-27T08:22:03.240 INFO:teuthology.orchestra.run.smithi097.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 812 kB/s | 248 kB 00:00 2024-03-27T08:22:03.325 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:07 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:03.390 INFO:teuthology.orchestra.run.smithi097.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2463.g022fc18a.e 310 kB/s | 141 kB 00:00 2024-03-27T08:22:03.476 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 870 k 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout:Install 2 Packages 2024-03-27T08:22:03.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:03.478 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 949 k 2024-03-27T08:22:03.478 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 4.0 M 2024-03-27T08:22:03.478 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:03.606 INFO:teuthology.orchestra.run.smithi190.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 616 kB/s | 79 kB 00:00 2024-03-27T08:22:03.665 INFO:teuthology.orchestra.run.smithi097.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 136 kB/s | 58 kB 00:00 2024-03-27T08:22:03.799 INFO:teuthology.orchestra.run.smithi097.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 125 kB/s | 51 kB 00:00 2024-03-27T08:22:03.932 INFO:teuthology.orchestra.run.smithi097.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 755 kB/s | 548 kB 00:00 2024-03-27T08:22:03.936 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:03.936 INFO:teuthology.orchestra.run.smithi097.stdout:Total 5.4 MB/s | 6.8 MB 00:01 2024-03-27T08:22:03.948 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:03.959 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:03.959 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:04.122 INFO:teuthology.orchestra.run.smithi190.stdout:(2/2): ceph-fuse-19.0.0-2463.g022fc18a.el9.x86_ 1.3 MB/s | 870 kB 00:00 2024-03-27T08:22:04.124 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:04.124 INFO:teuthology.orchestra.run.smithi190.stdout:Total 1.4 MB/s | 949 kB 00:00 2024-03-27T08:22:04.124 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:04.146 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:04.146 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:04.343 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:04.412 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:04.523 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-27T08:22:04.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-27T08:22:04.568 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-27T08:22:04.625 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:22:05.086 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-27T08:22:05.170 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-27T08:22:05.232 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-27T08:22:05.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:22:05.358 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:22:05.359 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/2 2024-03-27T08:22:05.685 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-27T08:22:05.685 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:05.685 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:05.685 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-27T08:22:05.685 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:05.685 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:05.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-volume 2024-03-27T08:22:06.069 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 6/6 2024-03-27T08:22:06.070 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 1/6 2024-03-27T08:22:06.070 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-27T08:22:06.070 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-27T08:22:06.070 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-27T08:22:06.070 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-27T08:22:06.373 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-27T08:22:06.373 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:06.373 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:06.373 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:22:06.373 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T08:22:06.374 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T08:22:06.374 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T08:22:06.374 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T08:22:06.374 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T08:22:06.374 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:06.374 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:06.515 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:10 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:06.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-fuse 2024-03-27T08:22:06.665 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:06.665 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:06.665 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 264 k 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout:Install 2 Packages 2024-03-27T08:22:06.666 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:06.667 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 332 k 2024-03-27T08:22:06.667 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 1.5 M 2024-03-27T08:22:06.668 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:06.787 INFO:teuthology.orchestra.run.smithi190.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 570 kB/s | 69 kB 00:00 2024-03-27T08:22:06.954 INFO:teuthology.orchestra.run.smithi190.stdout:(2/2): ceph-volume-19.0.0-2463.g022fc18a.el9.no 917 kB/s | 264 kB 00:00 2024-03-27T08:22:06.955 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:06.956 INFO:teuthology.orchestra.run.smithi190.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-03-27T08:22:06.956 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:06.964 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:06.964 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:07.006 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:07.006 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:07.206 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:12 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:07.357 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:07.357 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:07.357 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 870 k 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout:Install 2 Packages 2024-03-27T08:22:07.358 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:07.359 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 949 k 2024-03-27T08:22:07.359 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 4.0 M 2024-03-27T08:22:07.359 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:07.402 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-27T08:22:07.432 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:22:07.448 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:22:07.448 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:22:07.448 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T08:22:07.448 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:07.493 INFO:teuthology.orchestra.run.smithi097.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 586 kB/s | 79 kB 00:00 2024-03-27T08:22:07.752 INFO:teuthology.orchestra.run.smithi097.stdout:(2/2): ceph-fuse-19.0.0-2463.g022fc18a.el9.x86_ 2.2 MB/s | 870 kB 00:00 2024-03-27T08:22:07.753 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:07.753 INFO:teuthology.orchestra.run.smithi097.stdout:Total 2.3 MB/s | 949 kB 00:00 2024-03-27T08:22:07.753 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:07.775 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:07.775 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:07.848 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:07.849 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 1/2 2024-03-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-27T08:22:08.269 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:08.515 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:08.822 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install librados-devel 2024-03-27T08:22:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2/2 2024-03-27T08:22:08.965 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/2 2024-03-27T08:22:09.315 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-27T08:22:09.315 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:09.315 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:09.315 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-27T08:22:09.315 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:09.315 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:09.394 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:13 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:09.543 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:09.544 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:09.544 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-03-27T08:22:09.544 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 125 k 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 125 k 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 445 k 2024-03-27T08:22:09.545 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:09.586 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-volume 2024-03-27T08:22:10.003 INFO:teuthology.orchestra.run.smithi190.stdout:librados-devel-19.0.0-2463.g022fc18a.el9.x86_64 273 kB/s | 125 kB 00:00 2024-03-27T08:22:10.003 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:10.004 INFO:teuthology.orchestra.run.smithi190.stdout:Total 272 kB/s | 125 kB 00:00 2024-03-27T08:22:10.004 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:10.009 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:10.009 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:10.065 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:10.065 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:15 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:10.208 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:10.299 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-2463.g022fc18a.el9 ceph-noarch 264 k 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Install 2 Packages 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 332 k 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 1.5 M 2024-03-27T08:22:10.300 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:10.309 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:10.424 INFO:teuthology.orchestra.run.smithi097.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 560 kB/s | 69 kB 00:00 2024-03-27T08:22:10.719 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:10.999 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:10.999 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:10.999 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:10.999 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:10.999 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:10.999 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:11.065 INFO:teuthology.orchestra.run.smithi097.stdout:(2/2): ceph-volume-19.0.0-2463.g022fc18a.el9.no 345 kB/s | 264 kB 00:00 2024-03-27T08:22:11.067 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:11.067 INFO:teuthology.orchestra.run.smithi097.stdout:Total 433 kB/s | 332 kB 00:00 2024-03-27T08:22:11.067 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:11.076 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:11.076 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:11.118 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:11.118 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:11.258 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install libcephfs2 2024-03-27T08:22:11.342 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-27T08:22:11.568 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:22:11.583 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2/2 2024-03-27T08:22:11.584 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:22:11.584 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T08:22:11.584 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:11.827 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:15 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:11.928 INFO:teuthology.orchestra.run.smithi190.stdout:Package libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:11.983 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:11.984 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:11.984 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:12.054 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install libcephfs-devel 2024-03-27T08:22:12.314 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 1/2 2024-03-27T08:22:12.618 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:16 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-2463.g022fc18a.el9.noarch 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:12.672 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:12.766 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:12.768 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-03-27T08:22:12.768 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:12.768 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 32 k 2024-03-27T08:22:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:12.769 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:12.769 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:12.769 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:22:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:12.770 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 32 k 2024-03-27T08:22:12.770 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 142 k 2024-03-27T08:22:12.770 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:12.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librados-devel 2024-03-27T08:22:13.090 INFO:teuthology.orchestra.run.smithi190.stdout:libcephfs-devel-19.0.0-2463.g022fc18a.el9.x86_6 98 kB/s | 32 kB 00:00 2024-03-27T08:22:13.090 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:13.091 INFO:teuthology.orchestra.run.smithi190.stdout:Total 98 kB/s | 32 kB 00:00 2024-03-27T08:22:13.091 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:13.094 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:13.094 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:13.117 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:13.117 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:13.330 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:13.564 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:18 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:13.702 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:13.711 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 125 k 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 125 k 2024-03-27T08:22:13.712 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 445 k 2024-03-27T08:22:13.713 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:13.963 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:13.964 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:13.973 INFO:teuthology.orchestra.run.smithi097.stdout:librados-devel-19.0.0-2463.g022fc18a.el9.x86_64 480 kB/s | 125 kB 00:00 2024-03-27T08:22:13.973 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:13.974 INFO:teuthology.orchestra.run.smithi097.stdout:Total 478 kB/s | 125 kB 00:00 2024-03-27T08:22:13.974 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:13.980 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:13.980 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:14.036 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:14.036 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:14.146 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install librados2 2024-03-27T08:22:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:14.707 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:18 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:14.804 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:14.856 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:14.857 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:14.857 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:14.929 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install librbd1 2024-03-27T08:22:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:15.012 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:15.013 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:15.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install libcephfs2 2024-03-27T08:22:15.494 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:19 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:15.591 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:15.643 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:15.644 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:15.645 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:15.719 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-rados 2024-03-27T08:22:15.851 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:20 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:15.950 INFO:teuthology.orchestra.run.smithi097.stdout:Package libcephfs2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:16.003 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:16.004 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:16.004 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:16.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install libcephfs-devel 2024-03-27T08:22:16.277 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:20 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:16.374 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:16.426 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:16.427 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:16.427 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:16.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-rgw 2024-03-27T08:22:16.640 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:21 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:16.787 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 32 k 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 32 k 2024-03-27T08:22:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 142 k 2024-03-27T08:22:16.789 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:17.057 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:21 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:17.127 INFO:teuthology.orchestra.run.smithi097.stdout:libcephfs-devel-19.0.0-2463.g022fc18a.el9.x86_6 94 kB/s | 32 kB 00:00 2024-03-27T08:22:17.127 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:17.127 INFO:teuthology.orchestra.run.smithi097.stdout:Total 93 kB/s | 32 kB 00:00 2024-03-27T08:22:17.127 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:17.131 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:17.131 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:17.153 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:17.160 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:17.161 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:17.206 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:17.207 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:17.208 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:17.253 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:17.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-cephfs 2024-03-27T08:22:17.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:17.775 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:17.839 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:21 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:17.936 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:17.987 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:17.988 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:17.988 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:18.029 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:18.029 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:18.061 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install python3-rbd 2024-03-27T08:22:18.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librados2 2024-03-27T08:22:18.619 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:22 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:18.715 INFO:teuthology.orchestra.run.smithi190.stdout:Package python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:18.767 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:18.768 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-03-27T08:22:18.768 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:18.790 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:23 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:18.845 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install rbd-fuse 2024-03-27T08:22:18.885 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:18.936 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:18.937 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:18.938 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:19.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librbd1 2024-03-27T08:22:19.425 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:23 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:19.563 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:24 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:19.586 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 90 k 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:22:19.587 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:19.588 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 90 k 2024-03-27T08:22:19.588 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 313 k 2024-03-27T08:22:19.588 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:19.659 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:19.711 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:19.712 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:19.712 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:19.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rados 2024-03-27T08:22:19.935 INFO:teuthology.orchestra.run.smithi190.stdout:rbd-fuse-19.0.0-2463.g022fc18a.el9.x86_64.rpm 258 kB/s | 90 kB 00:00 2024-03-27T08:22:19.936 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:19.936 INFO:teuthology.orchestra.run.smithi190.stdout:Total 257 kB/s | 90 kB 00:00 2024-03-27T08:22:19.937 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:19.946 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:19.946 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:20.003 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:20.003 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:20.148 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:20.252 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:20.352 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:25 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:20.449 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rados-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:20.501 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:20.502 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:20.502 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:20.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rgw 2024-03-27T08:22:20.655 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:20.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:20.961 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:20.961 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:20.961 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:20.961 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:20.961 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:21.141 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:26 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:21.195 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install rbd-mirror 2024-03-27T08:22:21.238 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rgw-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:21.290 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:21.292 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:21.292 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:21.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-cephfs 2024-03-27T08:22:21.760 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:25 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:21.916 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:21.916 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repo Size 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 3.4 M 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 29 M 2024-03-27T08:22:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:21.934 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:26 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:22.031 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-cephfs-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:22.082 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:22.083 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:22.083 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:22.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rbd 2024-03-27T08:22:22.715 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:27 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:22.811 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rbd-2:19.0.0-2463.g022fc18a.el9.x86_64 is already installed. 2024-03-27T08:22:22.863 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:22.864 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-27T08:22:22.864 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:22.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-fuse 2024-03-27T08:22:23.280 INFO:teuthology.orchestra.run.smithi190.stdout:rbd-mirror-19.0.0-2463.g022fc18a.el9.x86_64.rpm 2.5 MB/s | 3.4 MB 00:01 2024-03-27T08:22:23.281 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:23.281 INFO:teuthology.orchestra.run.smithi190.stdout:Total 2.5 MB/s | 3.4 MB 00:01 2024-03-27T08:22:23.282 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:23.293 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:23.293 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:23.369 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:23.370 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:23.517 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:28 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:23.678 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:23.678 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 90 k 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 90 k 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 313 k 2024-03-27T08:22:23.679 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:23.939 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:23.998 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:24.023 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:24.023 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:22:24.023 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T08:22:24.024 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:22:24.024 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:22:24.024 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:24.197 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-fuse-19.0.0-2463.g022fc18a.el9.x86_64.rpm 173 kB/s | 90 kB 00:00 2024-03-27T08:22:24.198 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:24.198 INFO:teuthology.orchestra.run.smithi097.stdout:Total 173 kB/s | 90 kB 00:00 2024-03-27T08:22:24.198 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:24.207 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:24.207 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:24.265 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:24.266 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:24.410 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:24.514 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:24.947 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:25.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:25.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:25.219 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:25.219 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:25.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:25.219 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:25.277 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:25.277 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:25.277 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:25.277 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:25.277 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:25.277 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:25.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-mirror 2024-03-27T08:22:25.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install rbd-nbd 2024-03-27T08:22:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:31 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:26.113 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:30 ago on Wed 27 Mar 2024 08:19:56 AM UTC. 2024-03-27T08:22:26.188 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:26.188 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:26.188 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-27T08:22:26.188 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:26.188 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:26.188 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 3.4 M 2024-03-27T08:22:26.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:26.189 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:26.189 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:26.189 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:22:26.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:26.189 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 3.4 M 2024-03-27T08:22:26.190 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 29 M 2024-03-27T08:22:26.190 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:26.266 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 175 k 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout:Install 1 Package 2024-03-27T08:22:26.267 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:26.268 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 175 k 2024-03-27T08:22:26.268 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 673 k 2024-03-27T08:22:26.268 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-03-27T08:22:26.576 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-mirror-19.0.0-2463.g022fc18a.el9.x86_64.rpm 8.8 MB/s | 3.4 MB 00:00 2024-03-27T08:22:26.577 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:26.577 INFO:teuthology.orchestra.run.smithi097.stdout:Total 8.8 MB/s | 3.4 MB 00:00 2024-03-27T08:22:26.577 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:26.587 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:26.587 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:26.651 INFO:teuthology.orchestra.run.smithi190.stdout:rbd-nbd-19.0.0-2463.g022fc18a.el9.x86_64.rpm 457 kB/s | 175 kB 00:00 2024-03-27T08:22:26.652 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:26.652 INFO:teuthology.orchestra.run.smithi190.stdout:Total 455 kB/s | 175 kB 00:00 2024-03-27T08:22:26.652 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-03-27T08:22:26.663 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:26.663 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:26.664 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-03-27T08:22:26.664 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-03-27T08:22:26.718 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-03-27T08:22:26.719 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-03-27T08:22:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-03-27T08:22:26.975 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:27.216 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:27.285 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:27.309 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:27.309 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T08:22:27.309 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T08:22:27.309 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:22:27.309 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T08:22:27.310 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:27.456 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:27.752 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:27.753 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:27.753 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-03-27T08:22:27.753 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:27.753 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:27.753 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-03-27T08:22:28.031 DEBUG:teuthology.parallel:result is None 2024-03-27T08:22:28.568 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:28.568 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:28.568 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:28.568 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:28.568 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:28.568 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:28.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-nbd 2024-03-27T08:22:29.400 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:34 ago on Wed 27 Mar 2024 08:19:55 AM UTC. 2024-03-27T08:22:29.549 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-2463.g022fc18a.el9 ceph 175 k 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T08:22:29.550 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-27T08:22:29.551 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:29.551 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 175 k 2024-03-27T08:22:29.551 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 673 k 2024-03-27T08:22:29.551 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T08:22:29.780 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-nbd-19.0.0-2463.g022fc18a.el9.x86_64.rpm 764 kB/s | 175 kB 00:00 2024-03-27T08:22:29.780 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T08:22:29.781 INFO:teuthology.orchestra.run.smithi097.stdout:Total 759 kB/s | 175 kB 00:00 2024-03-27T08:22:29.781 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T08:22:29.790 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T08:22:29.790 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T08:22:29.847 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T08:22:29.847 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T08:22:29.999 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T08:22:30.104 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:30.518 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 1/1 2024-03-27T08:22:30.808 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:30.809 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T08:22:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-2463.g022fc18a.el9.x86_64 2024-03-27T08:22:30.809 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:30.809 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T08:22:31.051 DEBUG:teuthology.parallel:result is None 2024-03-27T08:22:31.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:22:31.279 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T08:22:31.300 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-2463.g022fc18a.el9 2024-03-27T08:22:31.301 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2463.g022fc18a.el9 2024-03-27T08:22:31.301 INFO:teuthology.task.install:The correct ceph version 19.0.0-2463.g022fc18a is installed. 2024-03-27T08:22:31.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=022fc18aa97a30a2f3803bcc4f7d8f3544371a09 2024-03-27T08:22:31.519 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T08:22:31.543 INFO:teuthology.orchestra.run.smithi190.stdout:19.0.0-2463.g022fc18a.el9 2024-03-27T08:22:31.543 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2463.g022fc18a.el9 2024-03-27T08:22:31.543 INFO:teuthology.task.install:The correct ceph version 19.0.0-2463.g022fc18a is installed. 2024-03-27T08:22:31.545 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-27T08:22:31.545 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:31.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T08:22:31.573 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:31.573 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T08:22:31.614 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-27T08:22:31.614 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:31.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T08:22:31.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T08:22:31.710 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:31.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T08:22:31.740 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T08:22:31.810 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-27T08:22:31.810 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:31.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T08:22:31.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T08:22:31.903 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:31.904 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T08:22:31.932 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T08:22:32.001 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-27T08:22:32.001 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:32.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T08:22:32.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T08:22:32.094 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:32.094 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T08:22:32.122 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T08:22:32.191 INFO:teuthology.run_tasks:Running task ceph... 2024-03-27T08:22:32.277 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-27T08:22:32.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-03-27T08:22:32.280 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-03-27T08:22:32.307 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-27T08:22:32.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-27T08:22:32.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-27T08:22:32.373 INFO:tasks.ceph:Creating extra log directories... 2024-03-27T08:22:32.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-27T08:22:32.413 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-27T08:22:32.448 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-27T08:22:32.448 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-27T08:22:32.448 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880', 'branch': 'wip-matanb-crimson-foreign-ptr', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7625880', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-foreign-ptr'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'eae914f097640d21427554e7f6aa6f1e9c4a7748'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 7338, 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_eae914f097640d21427554e7f6aa6f1e9c4a7748/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'eae914f097640d21427554e7f6aa6f1e9c4a7748', 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxONJN7jzLFKzFOxCdVzc2pZrWcixdRHg3DKGiCS7tp2vvp/OUOaK15GFzgk/QKd5LXJ61keemHHi3aHNLkbGo=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqxg1zoRJJjLS/Y+WaGnlKi1sat+8mnVePItmpNfEkGm/mIKFGUIDBacx6oEa13Ls9V4E+gXhM81ybdFzRl5QA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '022fc18aa97a30a2f3803bcc4f7d8f3544371a09', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-27_08:05:55', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-27T08:22:32.449 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-27T08:22:32.480 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-27T08:22:32.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-03-27T08:22:32.535 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-03-27T08:22:32.562 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:32.562 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T08:22:32.615 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T08:22:32.615 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-03-27T08:22:32.669 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 08:22:27.552467036 +0000 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 08:18:58.522804950 +0000 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 08:18:58.522804950 +0000 2024-03-27T08:22:32.670 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 08:18:58.522804950 +0000 2024-03-27T08:22:32.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T08:22:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T08:22:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T08:22:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00023519 s, 2.2 MB/s 2024-03-27T08:22:32.736 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T08:22:32.791 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-03-27T08:22:32.845 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 08:22:27.552467036 +0000 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 08:18:58.859799228 +0000 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 08:18:58.859799228 +0000 2024-03-27T08:22:32.846 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 08:18:58.859799228 +0000 2024-03-27T08:22:32.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T08:22:32.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T08:22:32.911 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T08:22:32.911 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000210939 s, 2.4 MB/s 2024-03-27T08:22:32.912 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T08:22:32.967 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-03-27T08:22:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T08:22:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:33.021 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-27T08:22:33.021 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 08:22:27.553467019 +0000 2024-03-27T08:22:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 08:18:59.197793489 +0000 2024-03-27T08:22:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 08:18:59.197793489 +0000 2024-03-27T08:22:33.022 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 08:18:59.197793489 +0000 2024-03-27T08:22:33.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T08:22:33.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T08:22:33.087 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T08:22:33.087 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000217367 s, 2.4 MB/s 2024-03-27T08:22:33.088 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T08:22:33.144 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 08:22:27.553467019 +0000 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 08:18:59.520788004 +0000 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 08:18:59.520788004 +0000 2024-03-27T08:22:33.199 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 08:18:59.520788004 +0000 2024-03-27T08:22:33.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T08:22:33.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T08:22:33.264 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T08:22:33.264 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000207524 s, 2.5 MB/s 2024-03-27T08:22:33.265 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T08:22:33.321 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-27T08:22:33.322 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:33.322 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T08:22:33.339 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T08:22:33.339 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-03-27T08:22:33.395 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-27 08:22:24.269438498 +0000 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-27 08:18:56.201451706 +0000 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-27 08:18:56.201451706 +0000 2024-03-27T08:22:33.396 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-03-27 08:18:56.201451706 +0000 2024-03-27T08:22:33.396 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T08:22:33.463 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-27T08:22:33.464 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-27T08:22:33.464 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000209523 s, 2.4 MB/s 2024-03-27T08:22:33.465 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T08:22:33.522 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-27 08:22:24.269438498 +0000 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-27 08:18:56.541441858 +0000 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-27 08:18:56.541441858 +0000 2024-03-27T08:22:33.579 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-03-27 08:18:56.541441858 +0000 2024-03-27T08:22:33.580 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T08:22:33.646 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-27T08:22:33.647 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-27T08:22:33.647 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000238693 s, 2.1 MB/s 2024-03-27T08:22:33.648 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T08:22:33.704 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-27 08:22:24.270438469 +0000 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-27 08:18:56.878432099 +0000 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-27 08:18:56.878432099 +0000 2024-03-27T08:22:33.760 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-03-27 08:18:56.878432099 +0000 2024-03-27T08:22:33.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T08:22:33.827 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-27T08:22:33.827 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-27T08:22:33.827 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000214233 s, 2.4 MB/s 2024-03-27T08:22:33.829 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T08:22:33.885 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-03-27T08:22:33.941 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T08:22:33.941 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T08:22:33.941 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-27T08:22:33.941 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T08:22:33.942 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T08:22:33.942 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-27 08:22:24.270438469 +0000 2024-03-27T08:22:33.942 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-27 08:18:57.225422051 +0000 2024-03-27T08:22:33.942 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-27 08:18:57.225422051 +0000 2024-03-27T08:22:33.942 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-03-27 08:18:57.225422051 +0000 2024-03-27T08:22:33.942 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T08:22:34.008 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-27T08:22:34.008 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-27T08:22:34.008 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000212711 s, 2.4 MB/s 2024-03-27T08:22:34.009 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T08:22:34.066 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-27T08:22:34.066 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-03-27T08:22:34.066 INFO:tasks.ceph:Generating config... 2024-03-27T08:22:34.067 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-03-27T08:22:34.068 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-03-27T08:22:34.068 INFO:tasks.ceph:[osd] debug monc = 20 2024-03-27T08:22:34.068 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-03-27T08:22:34.068 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-03-27T08:22:34.068 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-27T08:22:34.069 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-27T08:22:34.069 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-03-27T08:22:34.069 INFO:tasks.ceph:[osd] osd class default list = * 2024-03-27T08:22:34.069 INFO:tasks.ceph:[osd] osd class load list = * 2024-03-27T08:22:34.069 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-03-27T08:22:34.069 INFO:tasks.ceph:[client] debug ms = 1 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] ms client mode = crc 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-03-27T08:22:34.069 INFO:tasks.ceph:[global] ms service mode = crc 2024-03-27T08:22:34.069 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-27T08:22:34.070 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-27T08:22:34.070 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-03-27T08:22:34.070 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-03-27T08:22:34.070 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-03-27T08:22:34.070 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-03-27T08:22:34.070 INFO:tasks.ceph:Setting up mon.a... 2024-03-27T08:22:34.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-27T08:22:34.134 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2024-03-27T08:22:34.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-27T08:22:34.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-27T08:22:34.218 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97')] 2024-03-27T08:22:34.218 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-03-27T08:22:34.219 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:34.219 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-27T08:22:34.273 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:34.370+0000 7f33b8475f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:34.370+0000 7f33b8475f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:34.370+0000 7f33b8475f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:fsid ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2024-03-27T08:22:34.371818+0000 2024-03-27T08:22:34.363 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-03-27T08:22:34.371818+0000 2024-03-27T08:22:34.364 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 17 (quincy) 2024-03-27T08:22:34.364 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2024-03-27T08:22:34.364 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T08:22:34.364 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-03-27T08:22:34.365 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-27T08:22:34.379 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ab3c95de-2422-462d-839c-7edbd2518ef5... 2024-03-27T08:22:34.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-27T08:22:34.422 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-27T08:22:34.470 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-03-27T08:22:34.470 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-03-27T08:22:34.470 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-27T08:22:34.471 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-03-27T08:22:34.472 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.473 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = 172.21.15.97 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: ms client mode = crc 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: ms cluster mode = crc 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: ms mon client mode = crc 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: ms mon cluster mode = crc 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: ms mon service mode = crc 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: ms service mode = crc 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.474 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-03-27T08:22:34.475 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-03-27T08:22:34.476 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: debug monc = 20 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: crimson alien thread cpu cores = 6-7 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: crimson osd obc lru size = 10 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: osd blocked scrub grace period = 3600 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: osd class default list = * 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: osd class load list = * 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: osd shutdown pgref assert = True 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-03-27T08:22:34.477 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-03-27T08:22:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: mon min osdmap epochs = 50 2024-03-27T08:22:34.479 INFO:teuthology.orchestra.run.smithi097.stdout: mon osdmap full prune interval = 2 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: mon osdmap full prune min = 15 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: mon osdmap full prune txsize = 2 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: paxos service trim min = 10 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on pool no app = False 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default crimson = True 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-27T08:22:34.480 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-03-27T08:22:34.481 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-03-27T08:22:34.481 INFO:teuthology.orchestra.run.smithi097.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:22:34.481 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-03-27T08:22:34.481 INFO:teuthology.orchestra.run.smithi097.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:22:34.481 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi097.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi097.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-03-27T08:22:34.482 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.483 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-03-27T08:22:34.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = 172.21.15.97 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: ms client mode = crc 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: ms cluster mode = crc 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: ms mon client mode = crc 2024-03-27T08:22:34.485 INFO:teuthology.orchestra.run.smithi190.stdout: ms mon cluster mode = crc 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: ms mon service mode = crc 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: ms service mode = crc 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.486 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-03-27T08:22:34.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug monc = 20 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: crimson alien thread cpu cores = 6-7 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: crimson osd obc lru size = 10 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: osd blocked scrub grace period = 3600 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: osd class default list = * 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: osd class load list = * 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: osd shutdown pgref assert = True 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-03-27T08:22:34.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-03-27T08:22:34.489 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: mon min osdmap epochs = 50 2024-03-27T08:22:34.490 INFO:teuthology.orchestra.run.smithi190.stdout: mon osdmap full prune interval = 2 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: mon osdmap full prune min = 15 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: mon osdmap full prune txsize = 2 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: paxos service trim min = 10 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on pool no app = False 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default crimson = True 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2024-03-27T08:22:34.491 INFO:teuthology.orchestra.run.smithi190.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:22:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2024-03-27T08:22:34.492 INFO:teuthology.orchestra.run.smithi190.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:22:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2024-03-27T08:22:34.492 INFO:teuthology.orchestra.run.smithi190.stdout: crimson seastar cpu cores = 0-2 2024-03-27T08:22:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2024-03-27T08:22:34.492 INFO:teuthology.orchestra.run.smithi190.stdout: crimson seastar cpu cores = 3-5 2024-03-27T08:22:34.493 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-27T08:22:34.494 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-27T08:22:34.560 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-27T08:22:34.561 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:34.561 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-27T08:22:34.575 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:34.576 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-27T08:22:34.631 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:22:34.631 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:34.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-27T08:22:34.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-27T08:22:34.710 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:34.710 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:34.765 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:22:34.766 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:34.766 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-27T08:22:34.766 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-27T08:22:34.807 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:34.807 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:34.863 INFO:tasks.ceph:Setting up mon nodes... 2024-03-27T08:22:34.863 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-27T08:22:34.863 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-27T08:22:35.007 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-27T08:22:35.010 INFO:tasks.ceph:Setting up mds nodes... 2024-03-27T08:22:35.011 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-27T08:22:35.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-27T08:22:35.061 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-27T08:22:35.077 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-27T08:22:35.127 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-27T08:22:35.143 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-27T08:22:35.144 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-03-27T08:22:35.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-27T08:22:35.170 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-27T08:22:35.170 INFO:tasks.ceph:role: osd.0 2024-03-27T08:22:35.170 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:22:35.170 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:22:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:22:35.246 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:22:35.748 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-27T08:22:35.750 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-27T08:22:35.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-27T08:22:35.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-27T08:22:35.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-27T08:22:35.998 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-27T08:22:35.998 INFO:tasks.ceph:role: osd.1 2024-03-27T08:22:35.998 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:22:35.998 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:22:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:22:36.075 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:22:36.517 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-27T08:22:36.519 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-27T08:22:36.519 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-27T08:22:36.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-27T08:22:36.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:36.960 INFO:teuthology.orchestra.run.smithi097.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:36.961 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:36,970 seastar - Reactor backend: linux-aio 2024-03-27T08:22:36.977 INFO:teuthology.orchestra.run.smithi097.stderr:==28990==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:37,114 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:22:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:37,114 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-03-27T08:22:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:DEBUG 2024-03-27 08:22:37,115 [shard 0:main] seastar - reactor::drain 2024-03-27T08:22:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:Suppressions used: 2024-03-27T08:22:37.650 INFO:teuthology.orchestra.run.smithi097.stderr: count bytes template 2024-03-27T08:22:37.650 INFO:teuthology.orchestra.run.smithi097.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:37.650 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:22:37.711 INFO:teuthology.orchestra.run.smithi097.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:37.714 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:37,723 seastar - Reactor backend: linux-aio 2024-03-27T08:22:37.723 INFO:teuthology.orchestra.run.smithi097.stderr:==28986==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:41.176 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:41.177 INFO:teuthology.orchestra.run.smithi097.stderr:Suppressions used: 2024-03-27T08:22:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: count bytes template 2024-03-27T08:22:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:41.177 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:22:41.177 INFO:teuthology.orchestra.run.smithi097.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:41.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-27T08:22:41.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:41.519 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:41,528 seastar - Reactor backend: linux-aio 2024-03-27T08:22:41.524 INFO:teuthology.orchestra.run.smithi097.stderr:==29799==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99f029 0xb871af0 0xb87306c 0xb873282 0xb8733d8 0xb8738a1 0x54daf 0x36911 0x36965de 0x5da676f 0x5da6c46 0x5e02bd4 0x5d81750 0x5d81c84 0x5bc6c6f 0x5be1d33 0x6ebc2e4 0x6ecdab7 0x6ecef54 0x6ecefd9 0x6ecf0db 0x6ecf1d3 0x6ecf3e0 0x6ecf5c6 0x6ecf915 0x6ecfcb9 0x6ecfee9 0x6ed0153 0x6ed0420 0x6ed063e 0x6ed06f6 0x6ed07bc 0x6ed089d 0x5a2d276 0x5a2d380 0x5a2d43c 0x5a2d4f8 0x5a2d5b4 0x5a2d701 0x5a2d88c 0x5a2dc4d 0x5a4a6ca 0xb884775 0xb89ea0f 0xb93fa0d 0xb94105b 0xb61d7c3 0xb61e13f 0x37df266 0x37dfdbf 0x367fd7a 0x3feaf 0x3ff5f 0x346c444 2024-03-27T08:22:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:kernel callstack: 2024-03-27T08:22:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:41,651 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:22:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:41,651 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-03-27T08:22:41.643 INFO:teuthology.orchestra.run.smithi097.stderr:DEBUG 2024-03-27 08:22:41,652 [shard 0:main] seastar - reactor::drain 2024-03-27T08:22:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:42.168 INFO:teuthology.orchestra.run.smithi097.stderr:Suppressions used: 2024-03-27T08:22:42.168 INFO:teuthology.orchestra.run.smithi097.stderr: count bytes template 2024-03-27T08:22:42.168 INFO:teuthology.orchestra.run.smithi097.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:42.168 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:42.168 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:22:42.225 INFO:teuthology.orchestra.run.smithi097.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:INFO 2024-03-27 08:22:42,238 seastar - Reactor backend: linux-aio 2024-03-27T08:22:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:==29797==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:45.708 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:45.709 INFO:teuthology.orchestra.run.smithi097.stderr:Suppressions used: 2024-03-27T08:22:45.709 INFO:teuthology.orchestra.run.smithi097.stderr: count bytes template 2024-03-27T08:22:45.709 INFO:teuthology.orchestra.run.smithi097.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:45.709 INFO:teuthology.orchestra.run.smithi097.stderr:----------------------------------------------------- 2024-03-27T08:22:45.709 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:22:45.709 INFO:teuthology.orchestra.run.smithi097.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:45.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-27T08:22:45.759 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-27T08:22:45.791 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-27T08:22:45.791 INFO:tasks.ceph:role: osd.2 2024-03-27T08:22:45.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:22:45.792 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:22:45.869 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:22:46.364 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-03-27T08:22:46.366 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-03-27T08:22:46.367 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-03-27T08:22:46.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-27T08:22:46.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-27T08:22:46.601 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-27T08:22:46.601 INFO:tasks.ceph:role: osd.3 2024-03-27T08:22:46.601 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:22:46.601 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-27T08:22:46.677 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-27T08:22:46.677 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-27T08:22:46.677 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-27T08:22:46.677 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-27T08:22:46.677 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-27T08:22:46.678 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-03-27T08:22:46.678 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-27T08:22:46.678 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-27T08:22:46.678 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-27T08:22:46.678 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-27T08:22:47.134 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-03-27T08:22:47.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-03-27T08:22:47.136 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-03-27T08:22:47.230 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-27T08:22:47.260 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:47.579 INFO:teuthology.orchestra.run.smithi190.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:47.580 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:47,588 seastar - Reactor backend: linux-aio 2024-03-27T08:22:47.600 INFO:teuthology.orchestra.run.smithi190.stderr:==29058==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:47.706 INFO:teuthology.orchestra.run.smithi190.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99f029 0xb871af0 0xb87306c 0xb873282 0xb8733d8 0xb8738a1 0x54daf 0x687b0 0x59cdd83 0x5a1f087 0x5e02be1 0x5d81750 0x5d81c84 0x5bc6c6f 0x5be1d33 0x6ebc2e4 0x6ecdab7 0x6ecef54 0x6ecefd9 0x6ecf0db 0x6ecf1d3 0x6ecf3e0 0x6ecf5c6 0x6ecf915 0x6ecfcb9 0x6ecfee9 0x6ed0153 0x6ed0420 0x6ed063e 0x6ed06f6 0x6ed07bc 0x6ed089d 0x5a2d276 0x5a2d380 0x5a2d43c 0x5a2d4f8 0x5a2d5b4 0x5a2d701 0x5a2d88c 0x5a2dc4d 0x5a4a6ca 0xb884775 0xb89ea0f 0xb93fa0d 0xb94105b 0xb61d7c3 0xb61e13f 0x37df266 0x37dfdbf 0x367fd7a 0x3feaf 0x3ff5f 0x346c444 2024-03-27T08:22:47.707 INFO:teuthology.orchestra.run.smithi190.stderr:kernel callstack: 2024-03-27T08:22:47.721 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:47,730 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:22:47.721 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:47,730 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-03-27T08:22:47.722 INFO:teuthology.orchestra.run.smithi190.stderr:DEBUG 2024-03-27 08:22:47,731 [shard 0:main] seastar - reactor::drain 2024-03-27T08:22:48.286 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:48.287 INFO:teuthology.orchestra.run.smithi190.stderr:Suppressions used: 2024-03-27T08:22:48.287 INFO:teuthology.orchestra.run.smithi190.stderr: count bytes template 2024-03-27T08:22:48.287 INFO:teuthology.orchestra.run.smithi190.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:48.287 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:48.287 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-03-27T08:22:48.341 INFO:teuthology.orchestra.run.smithi190.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:48.345 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:48,354 seastar - Reactor backend: linux-aio 2024-03-27T08:22:48.355 INFO:teuthology.orchestra.run.smithi190.stderr:==29054==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:51.803 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:51.803 INFO:teuthology.orchestra.run.smithi190.stderr:Suppressions used: 2024-03-27T08:22:51.803 INFO:teuthology.orchestra.run.smithi190.stderr: count bytes template 2024-03-27T08:22:51.803 INFO:teuthology.orchestra.run.smithi190.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:51.803 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:51.803 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-03-27T08:22:51.804 INFO:teuthology.orchestra.run.smithi190.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:51.829 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-27T08:22:51.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:52.142 INFO:teuthology.orchestra.run.smithi190.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:52.143 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:52,151 seastar - Reactor backend: linux-aio 2024-03-27T08:22:52.148 INFO:teuthology.orchestra.run.smithi190.stderr:==29867==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:52.267 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:52,275 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-27T08:22:52.267 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:52,275 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-03-27T08:22:52.267 INFO:teuthology.orchestra.run.smithi190.stderr:DEBUG 2024-03-27 08:22:52,276 [shard 0:main] seastar - reactor::drain 2024-03-27T08:22:52.824 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:52.824 INFO:teuthology.orchestra.run.smithi190.stderr:Suppressions used: 2024-03-27T08:22:52.824 INFO:teuthology.orchestra.run.smithi190.stderr: count bytes template 2024-03-27T08:22:52.824 INFO:teuthology.orchestra.run.smithi190.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:52.824 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:52.824 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-03-27T08:22:52.877 INFO:teuthology.orchestra.run.smithi190.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-27T08:22:52.880 INFO:teuthology.orchestra.run.smithi190.stderr:INFO 2024-03-27 08:22:52,889 seastar - Reactor backend: linux-aio 2024-03-27T08:22:52.895 INFO:teuthology.orchestra.run.smithi190.stderr:==29865==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stderr:Suppressions used: 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stderr: count bytes template 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stderr: 1 8 MallocExtension::Register 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stderr:----------------------------------------------------- 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-03-27T08:22:56.353 INFO:teuthology.orchestra.run.smithi190.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:22:56.376 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-27T08:22:56.404 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-27T08:22:56.404 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:56.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-27T08:22:56.436 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:56.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-27T08:22:56.508 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:56.508 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-27T08:22:56.536 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:56.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-27T08:22:56.604 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:56.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-27T08:22:56.670 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:56.670 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-27T08:22:56.688 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:56.688 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-27T08:22:56.724 INFO:tasks.ceph:Adding keys to all mons... 2024-03-27T08:22:56.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-27T08:22:56.757 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-03-27T08:22:56.757 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBN1wNmLf1PNBAANiiTeAIS/LUkAwCx4FiySQ== 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBS1wNmJS9dFxAAK8XSbWzChho2pfOhSEug4g== 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBL1wNmWn39ABAAIrgAy0zerBJwJJddZB0u0w== 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBY1wNm45k6HhAAMSlaSoZPMqtbyb1qZlkeYQ== 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBd1wNmIwcGAxAAJcy5s8hBYorCcJBzVtnWKA== 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBL1wNmnNo1BBAA60jsqeNPqGr84yB1CbjKHg== 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-03-27T08:22:56.758 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBL1wNmJEooCBAAhJr/rInVDXY0hQ2HILb/4g== 2024-03-27T08:22:56.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:22:56.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:22:56.933 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-27T08:22:56.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:22:57.056 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-27T08:22:57.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-27T08:22:57.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-27T08:22:57.248 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-27T08:22:57.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-27T08:22:57.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-27T08:22:57.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.397+0000 7f491b264a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.397+0000 7f491b264a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-27T08:22:57.430 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-27T08:22:57.485 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-27T08:22:57.485 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-27T08:22:57.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-27T08:22:57.527 INFO:tasks.ceph.mon.a:Started 2024-03-27T08:22:57.527 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-27T08:22:57.528 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-27T08:22:57.528 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-27T08:22:57.531 INFO:tasks.ceph.mgr.x:Started 2024-03-27T08:22:57.531 INFO:tasks.ceph:Setting crush tunables to default 2024-03-27T08:22:57.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2024-03-27T08:22:57.610 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-27T08:22:57.618+0000 7f323702ea80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.611 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-27T08:22:57.618+0000 7f323702ea80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.611 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-27T08:22:57.618+0000 7f323702ea80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.626 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:57.634+0000 7f3dfdd70000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.642+0000 7f6db5c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.649+0000 7f6db5c2d640 1 Processor -- start 2024-03-27T08:22:57.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.650+0000 7f6db5c2d640 1 -- start start 2024-03-27T08:22:57.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6db5c2d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:57.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6db5c2d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db009ee40 con 0x7f6db0134690 2024-03-27T08:22:57.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6db5c2d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f6db00736e0 con 0x7f6db0131730 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55820/0 (socket says 172.21.15.97:55820) 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 -- 172.21.15.97:0/171574939 learned_addr learned my addr 172.21.15.97:0/171574939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6da6ffd640 1 --1- 172.21.15.97:0/171574939 >> v1:172.21.15.97:6789/0 conn(0x7f6db0134690 0x7f6db0134aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42808/0 (socket says 172.21.15.97:42808) 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 -- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 msgr2=0x7f6db0131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 -- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 msgr2=0x7f6db0131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 --2- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-27T08:22:57.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.651+0000 7f6daf7fe640 1 --2- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T08:22:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676730958 0 0) 0x7f6db009ee40 con 0x7f6db0134690 2024-03-27T08:22:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d9c003580 con 0x7f6db0134690 2024-03-27T08:22:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2357900367 0 0) 0x7f6d9c003580 con 0x7f6db0134690 2024-03-27T08:22:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 msgr2=0x7f6db0131b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 --2- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db00a0850 con 0x7f6db0134690 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 205+0+0 (unknown 1545890108 0 0) 0x7f6d980015a0 con 0x7f6db0134690 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 >> v1:172.21.15.97:6789/0 conn(0x7f6db0134690 legacy=0x7f6db0134aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 --2- 172.21.15.97:0/171574939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6d9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6db00736e0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.675+0000 7f6daf7fe640 1 --2- 172.21.15.97:0/171574939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6d9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.676+0000 7f6daf7fe640 1 -- 172.21.15.97:0/171574939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c003580 con 0x7f6d9c0039d0 2024-03-27T08:22:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.677+0000 7f6daf7fe640 1 --2- 172.21.15.97:0/171574939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6d9c003e20 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f66f96ac599c73 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.677+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6da0009680 con 0x7f6d9c0039d0 2024-03-27T08:22:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.677+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6da000c040 con 0x7f6d9c0039d0 2024-03-27T08:22:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.677+0000 7f6daeffd640 1 -- 172.21.15.97:0/171574939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6da0009cf0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 -- 172.21.15.97:0/171574939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 msgr2=0x7f6d9c003e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 --2- 172.21.15.97:0/171574939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6d9c003e20 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 -- 172.21.15.97:0/171574939 shutdown_connections 2024-03-27T08:22:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 --2- 172.21.15.97:0/171574939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6d9c003e20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 --2- 172.21.15.97:0/171574939 >> v2:172.21.15.97:3300/0 conn(0x7f6db0131730 0x7f6db0131b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 -- 172.21.15.97:0/171574939 >> 172.21.15.97:0/171574939 conn(0x7f6db00a4ae0 msgr2=0x7f6db00a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 -- 172.21.15.97:0/171574939 shutdown_connections 2024-03-27T08:22:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.678+0000 7f6db5c2d640 1 -- 172.21.15.97:0/171574939 wait complete. 2024-03-27T08:22:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.679+0000 7f6db5c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.672 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:57.679+0000 7f3dfdd70000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.673 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:57.679+0000 7f3dfdd70000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.680+0000 7f6db5c2d640 1 Processor -- start 2024-03-27T08:22:57.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.680+0000 7f6db5c2d640 1 -- start start 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.681+0000 7f6db5c2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6db011d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.681+0000 7f6db5c2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6db011ece0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.681+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6db011d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.681+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6db011d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55844/0 (socket says 172.21.15.97:55844) 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.681+0000 7f6daf7fe640 1 -- 172.21.15.97:0/4098284240 learned_addr learned my addr 172.21.15.97:0/4098284240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.681+0000 7f6daf7fe640 1 -- 172.21.15.97:0/4098284240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db01202d0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6daf7fe640 1 --2- 172.21.15.97:0/4098284240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6db011d230 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6da0016e40 con 0x7f6d9c0039d0 2024-03-27T08:22:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6da0009bc0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db011ffe0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6da0017440 con 0x7f6d9c0039d0 2024-03-27T08:22:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db011f290 con 0x7f6d9c0039d0 2024-03-27T08:22:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f6da0017980 con 0x7f6d9c0039d0 2024-03-27T08:22:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.682+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (crc 0 0 0) 0x7f6da001c040 con 0x7f6d9c0039d0 2024-03-27T08:22:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.683+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db0063450 con 0x7f6d9c0039d0 2024-03-27T08:22:57.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.685+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f6da000f040 con 0x7f6d9c0039d0 2024-03-27T08:22:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.749+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6db0068ae0 con 0x7f6d9c0039d0 2024-03-27T08:22:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.750+0000 7f6dacff9640 1 -- 172.21.15.97:0/4098284240 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (crc 0 0 0) 0x7f6da0014410 con 0x7f6d9c0039d0 2024-03-27T08:22:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 msgr2=0x7f6db011d230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 --2- 172.21.15.97:0/4098284240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6db011d230 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 shutdown_connections 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 --2- 172.21.15.97:0/4098284240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c0039d0 0x7f6db011d230 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 >> 172.21.15.97:0/4098284240 conn(0x7f6db00a4ae0 msgr2=0x7f6db01ba6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 shutdown_connections 2024-03-27T08:22:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.751+0000 7f6db5c2d640 1 -- 172.21.15.97:0/4098284240 wait complete. 2024-03-27T08:22:57.754 INFO:tasks.ceph:check_enable_crimson: True 2024-03-27T08:22:57.754 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-03-27T08:22:57.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-03-27T08:22:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.864+0000 7fbf6b4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.870+0000 7fbf6b4ad640 1 Processor -- start 2024-03-27T08:22:57.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.870+0000 7fbf6b4ad640 1 -- start start 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf6b4ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf641378c0 0x7fbf64139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf6b4ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf640a06a0 con 0x7fbf64131650 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf6b4ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fbf64132170 con 0x7fbf641378c0 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf69249640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbf64131650 0x7fbf64131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42824/0 (socket says 172.21.15.97:42824) 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf69249640 1 -- 172.21.15.97:0/3890689549 learned_addr learned my addr 172.21.15.97:0/3890689549 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf68a48640 1 --2- 172.21.15.97:0/3890689549 >> v2:172.21.15.97:3300/0 conn(0x7fbf641378c0 0x7fbf64139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf5bfff640 1 -- 172.21.15.97:0/3890689549 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886819842 0 0) 0x7fbf640a06a0 con 0x7fbf64131650 2024-03-27T08:22:57.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf5bfff640 1 -- 172.21.15.97:0/3890689549 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf50003580 con 0x7fbf64131650 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.871+0000 7fbf68a48640 1 -- 172.21.15.97:0/3890689549 >> v1:172.21.15.97:6789/0 conn(0x7fbf64131650 legacy=0x7fbf64131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf68a48640 1 -- 172.21.15.97:0/3890689549 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6406de30 con 0x7fbf641378c0 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf68a48640 1 --2- 172.21.15.97:0/3890689549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf641378c0 0x7fbf64139ce0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2073239a4d15fd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf5bfff640 1 -- 172.21.15.97:0/3890689549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbf54009d90 con 0x7fbf641378c0 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf5bfff640 1 -- 172.21.15.97:0/3890689549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbf5400e230 con 0x7fbf641378c0 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf5bfff640 1 -- 172.21.15.97:0/3890689549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbf5400e4d0 con 0x7fbf641378c0 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/3890689549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf641378c0 msgr2=0x7fbf64139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.872+0000 7fbf6b4ad640 1 --2- 172.21.15.97:0/3890689549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf641378c0 0x7fbf64139ce0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.873+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/3890689549 shutdown_connections 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.873+0000 7fbf6b4ad640 1 --2- 172.21.15.97:0/3890689549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf641378c0 0x7fbf64139ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.873+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/3890689549 >> 172.21.15.97:0/3890689549 conn(0x7fbf640a15e0 msgr2=0x7fbf640a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.873+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/3890689549 shutdown_connections 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.873+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/3890689549 wait complete. 2024-03-27T08:22:57.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.873+0000 7fbf6b4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.874+0000 7fbf6b4ad640 1 Processor -- start 2024-03-27T08:22:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.874+0000 7fbf6b4ad640 1 -- start start 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.875+0000 7fbf6b4ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 0x7fbf6411e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.875+0000 7fbf6b4ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6411e070 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.875+0000 7fbf69249640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 0x7fbf6411e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.875+0000 7fbf69249640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 0x7fbf6411e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55850/0 (socket says 172.21.15.97:55850) 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.875+0000 7fbf69249640 1 -- 172.21.15.97:0/1978551789 learned_addr learned my addr 172.21.15.97:0/1978551789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf69249640 1 -- 172.21.15.97:0/1978551789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf64120e50 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf69249640 1 --2- 172.21.15.97:0/1978551789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 0x7fbf6411e4f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbf60005a60 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbf60005bc0 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf64121bb0 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbf60005e60 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641218b0 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fbf60007440 con 0x7fbf64131650 2024-03-27T08:22:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.876+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (crc 0 0 0) 0x7fbf6000e910 con 0x7fbf64131650 2024-03-27T08:22:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.877+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf64073320 con 0x7fbf64131650 2024-03-27T08:22:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.879+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fbf6000ead0 con 0x7fbf64131650 2024-03-27T08:22:57.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:57.951+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) -- 0x7fbf64063450 con 0x7fbf64131650 2024-03-27T08:22:57.944 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-27T08:22:57.952+0000 7f322d9f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-03-27T08:22:57.944 INFO:tasks.ceph.mon.a.smithi097.stderr:Please be aware that this feature is experimental, untested, 2024-03-27T08:22:57.944 INFO:tasks.ceph.mon.a.smithi097.stderr:unsupported, and may result in data corruption, data loss, 2024-03-27T08:22:57.944 INFO:tasks.ceph.mon.a.smithi097.stderr:and/or irreparable damage to your cluster. Do not use 2024-03-27T08:22:57.944 INFO:tasks.ceph.mon.a.smithi097.stderr:feature with important data. 2024-03-27T08:22:57.945 INFO:tasks.ceph.mon.a.smithi097.stderr: 2024-03-27T08:22:58.152 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.160+0000 7f3dfdd70000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T08:22:58.205 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.213+0000 7f3dfdd70000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T08:22:58.736 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.744+0000 7f3dfdd70000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T08:22:58.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.750+0000 7fbf59ffb640 1 -- 172.21.15.97:0/1978551789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) ==== 99+0+0 (crc 0 0 0) 0x7fbf6000ed40 con 0x7fbf64131650 2024-03-27T08:22:58.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.751+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 msgr2=0x7fbf6411e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:58.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.751+0000 7fbf6b4ad640 1 --2- 172.21.15.97:0/1978551789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 0x7fbf6411e4f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:58.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.752+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 shutdown_connections 2024-03-27T08:22:58.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.752+0000 7fbf6b4ad640 1 --2- 172.21.15.97:0/1978551789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf64131650 0x7fbf6411e4f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:58.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.752+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 >> 172.21.15.97:0/1978551789 conn(0x7fbf640a15e0 msgr2=0x7fbf641358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:58.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.752+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 shutdown_connections 2024-03-27T08:22:58.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:22:58.752+0000 7fbf6b4ad640 1 -- 172.21.15.97:0/1978551789 wait complete. 2024-03-27T08:22:58.756 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-27T08:22:58.757 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:58.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-27T08:22:58.785 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:22:58.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-27T08:22:58.788 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.796+0000 7f3dfdd70000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T08:22:58.832 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.840+0000 7f3dfdd70000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T08:22:58.853 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:58.853 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-27T08:22:58.879 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-27T08:22:58.880 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-27T08:22:58.888 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.896+0000 7f3dfdd70000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T08:22:58.914 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 8aee0ca6-e299-4707-bb1a-9d92e6da31c2 0 2024-03-27T08:22:58.933 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:58.941+0000 7f3dfdd70000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T08:22:59.042 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.044+0000 7f2fd366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.043 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.050+0000 7f2fd366d640 1 Processor -- start 2024-03-27T08:22:59.043 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.050+0000 7f2fd366d640 1 -- start start 2024-03-27T08:22:59.043 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd366d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fcc113670 0x7f2fcc113a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.043 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd366d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fcc114240 con 0x7f2fcc1180e0 2024-03-27T08:22:59.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd366d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2fcc118c20 con 0x7f2fcc113670 2024-03-27T08:22:59.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd1409640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fcc113670 0x7f2fcc113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd0c08640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2fcc1180e0 0x7f2fcc1184f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.190:38308/0 (socket says 172.21.15.190:38308) 2024-03-27T08:22:59.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd1409640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fcc113670 0x7f2fcc113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39782/0 (socket says 172.21.15.190:39782) 2024-03-27T08:22:59.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd0c08640 1 -- 172.21.15.190:0/905947778 learned_addr learned my addr 172.21.15.190:0/905947778 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-03-27T08:22:59.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd1409640 1 -- 172.21.15.190:0/905947778 >> v1:172.21.15.97:6789/0 conn(0x7f2fcc1180e0 legacy=0x7f2fcc1184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.051+0000 7f2fd1409640 1 -- 172.21.15.190:0/905947778 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc118d90 con 0x7f2fcc113670 2024-03-27T08:22:59.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.052+0000 7f2fd1409640 1 --2- 172.21.15.190:0/905947778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc113a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881d3f31cfcceff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.052+0000 7f2fc3fff640 1 -- 172.21.15.190:0/905947778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2fbc009d90 con 0x7f2fcc113670 2024-03-27T08:22:59.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.052+0000 7f2fc3fff640 1 -- 172.21.15.190:0/905947778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2fbc00e230 con 0x7f2fcc113670 2024-03-27T08:22:59.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.052+0000 7f2fc3fff640 1 -- 172.21.15.190:0/905947778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2fbc00e4d0 con 0x7f2fcc113670 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 -- 172.21.15.190:0/905947778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 msgr2=0x7f2fcc113a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 --2- 172.21.15.190:0/905947778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc113a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 -- 172.21.15.190:0/905947778 shutdown_connections 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 --2- 172.21.15.190:0/905947778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc113a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 -- 172.21.15.190:0/905947778 >> 172.21.15.190:0/905947778 conn(0x7f2fcc06dab0 msgr2=0x7f2fcc06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 -- 172.21.15.190:0/905947778 shutdown_connections 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 1 -- 172.21.15.190:0/905947778 wait complete. 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.053+0000 7f2fd366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd366d640 1 Processor -- start 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd366d640 1 -- start start 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd366d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc1aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd366d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ae4a0 con 0x7f2fcc113670 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd1409640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc1aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd1409640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc1aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39784/0 (socket says 172.21.15.190:39784) 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd1409640 1 -- 172.21.15.190:0/234268842 learned_addr learned my addr 172.21.15.190:0/234268842 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.054+0000 7f2fd1409640 1 -- 172.21.15.190:0/234268842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1ae5e0 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fd1409640 1 --2- 172.21.15.190:0/234268842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc1aded0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2fbc014750 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2fbc0148b0 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc1ae7d0 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2fbc014b50 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a8230 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f2fbc014d50 con 0x7f2fcc113670 2024-03-27T08:22:59.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.055+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 747+0+0 (crc 0 0 0) 0x7f2fbc019040 con 0x7f2fcc113670 2024-03-27T08:22:59.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.056+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc1ae9c0 con 0x7f2fcc113670 2024-03-27T08:22:59.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.058+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f2fbc011380 con 0x7f2fcc113670 2024-03-27T08:22:59.057 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.065+0000 7f3dfdd70000 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T08:22:59.123 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.131+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8aee0ca6-e299-4707-bb1a-9d92e6da31c2", "id": 0} v 0) -- 0x7f2fcc118180 con 0x7f2fcc113670 2024-03-27T08:22:59.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.132+0000 7f2fc1ffb640 1 -- 172.21.15.190:0/234268842 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8aee0ca6-e299-4707-bb1a-9d92e6da31c2", "id": 0}]=0 v4) ==== 112+0+2 (crc 0 0 0) 0x7f2fbc011520 con 0x7f2fcc113670 2024-03-27T08:22:59.124 INFO:teuthology.orchestra.run.smithi190.stdout:0 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 msgr2=0x7f2fcc1aded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 --2- 172.21.15.190:0/234268842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc1aded0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 shutdown_connections 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 --2- 172.21.15.190:0/234268842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fcc113670 0x7f2fcc1aded0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 >> 172.21.15.190:0/234268842 conn(0x7f2fcc06dab0 msgr2=0x7f2fcc1170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 shutdown_connections 2024-03-27T08:22:59.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.133+0000 7f2fd366d640 1 -- 172.21.15.190:0/234268842 wait complete. 2024-03-27T08:22:59.130 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.138+0000 7f3dfdd70000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T08:22:59.136 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ea23b051-821d-4838-b8ef-1cb78fa81d90 1 2024-03-27T08:22:59.179 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.187+0000 7f3dfdd70000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T08:22:59.236 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.244+0000 7f3dfdd70000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T08:22:59.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.248+0000 7f65ee314640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65ee314640 1 Processor -- start 2024-03-27T08:22:59.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65ee314640 1 -- start start 2024-03-27T08:22:59.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65ee314640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65e8113670 0x7f65e8113a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65ee314640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e8114240 con 0x7f65e81180e0 2024-03-27T08:22:59.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65ee314640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f65e8118c20 con 0x7f65e8113670 2024-03-27T08:22:59.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65e7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65e8113670 0x7f65e8113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65e7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65e8113670 0x7f65e8113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39788/0 (socket says 172.21.15.190:39788) 2024-03-27T08:22:59.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.255+0000 7f65e7fff640 1 -- 172.21.15.190:0/3435758347 learned_addr learned my addr 172.21.15.190:0/3435758347 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-27T08:22:59.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.256+0000 7f65e7fff640 1 -- 172.21.15.190:0/3435758347 >> v1:172.21.15.97:6789/0 conn(0x7f65e81180e0 legacy=0x7f65e81184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.256+0000 7f65e7fff640 1 -- 172.21.15.190:0/3435758347 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e8118d90 con 0x7f65e8113670 2024-03-27T08:22:59.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.256+0000 7f65e7fff640 1 --2- 172.21.15.190:0/3435758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e8113a80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b159ad0e28c532 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.256+0000 7f65e6ffd640 1 -- 172.21.15.190:0/3435758347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f65d8009e50 con 0x7f65e8113670 2024-03-27T08:22:59.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.256+0000 7f65e6ffd640 1 -- 172.21.15.190:0/3435758347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f65d800e230 con 0x7f65e8113670 2024-03-27T08:22:59.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.256+0000 7f65e6ffd640 1 -- 172.21.15.190:0/3435758347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f65d800e4d0 con 0x7f65e8113670 2024-03-27T08:22:59.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 -- 172.21.15.190:0/3435758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 msgr2=0x7f65e8113a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 --2- 172.21.15.190:0/3435758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e8113a80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 -- 172.21.15.190:0/3435758347 shutdown_connections 2024-03-27T08:22:59.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 --2- 172.21.15.190:0/3435758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e8113a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 -- 172.21.15.190:0/3435758347 >> 172.21.15.190:0/3435758347 conn(0x7f65e806dab0 msgr2=0x7f65e806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 -- 172.21.15.190:0/3435758347 shutdown_connections 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 1 -- 172.21.15.190:0/3435758347 wait complete. 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.257+0000 7f65ee314640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.258+0000 7f65ee314640 1 Processor -- start 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.258+0000 7f65ee314640 1 -- start start 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.258+0000 7f65ee314640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e81aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.258+0000 7f65ee314640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81ae4a0 con 0x7f65e8113670 2024-03-27T08:22:59.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.258+0000 7f65e7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e81aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.258+0000 7f65e7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e81aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39798/0 (socket says 172.21.15.190:39798) 2024-03-27T08:22:59.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e7fff640 1 -- 172.21.15.190:0/2701532563 learned_addr learned my addr 172.21.15.190:0/2701532563 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e7fff640 1 -- 172.21.15.190:0/2701532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e81ae5e0 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e7fff640 1 --2- 172.21.15.190:0/2701532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e81aded0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f65d800eea0 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f65d8015420 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f65d80156c0 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e81ae7d0 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81a8230 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f65d8015c50 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.259+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 862+0+0 (crc 0 0 0) 0x7f65d8011740 con 0x7f65e8113670 2024-03-27T08:22:59.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.260+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e81ae9c0 con 0x7f65e8113670 2024-03-27T08:22:59.254 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.262+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f65d80119a0 con 0x7f65e8113670 2024-03-27T08:22:59.286 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.294+0000 7f3dfdd70000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T08:22:59.327 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.335+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea23b051-821d-4838-b8ef-1cb78fa81d90", "id": 1} v 0) -- 0x7f65e80008d0 con 0x7f65e8113670 2024-03-27T08:22:59.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.336+0000 7f65e4ff9640 1 -- 172.21.15.190:0/2701532563 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea23b051-821d-4838-b8ef-1cb78fa81d90", "id": 1}]=0 v5) ==== 112+0+2 (crc 0 0 0) 0x7f65d8011bc0 con 0x7f65e8113670 2024-03-27T08:22:59.329 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 msgr2=0x7f65e81aded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 --2- 172.21.15.190:0/2701532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e81aded0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 shutdown_connections 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 --2- 172.21.15.190:0/2701532563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e8113670 0x7f65e81aded0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 >> 172.21.15.190:0/2701532563 conn(0x7f65e806dab0 msgr2=0x7f65e81170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 shutdown_connections 2024-03-27T08:22:59.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.337+0000 7f65ee314640 1 -- 172.21.15.190:0/2701532563 wait complete. 2024-03-27T08:22:59.340 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new b1d0dc27-69b7-42ad-99fb-50b882a4bfa3 2 2024-03-27T08:22:59.491 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.493+0000 7f2f9be2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.499+0000 7f2f9be2d640 1 Processor -- start 2024-03-27T08:22:59.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.499+0000 7f2f9be2d640 1 -- start start 2024-03-27T08:22:59.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f9be2d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f94113670 0x7f2f94113a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f9be2d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f94114240 con 0x7f2f941180e0 2024-03-27T08:22:59.492 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f9be2d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2f94118c20 con 0x7f2f94113670 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f99bc9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f94113670 0x7f2f94113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f993c8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2f941180e0 0x7f2f941184f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.190:38322/0 (socket says 172.21.15.190:38322) 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f99bc9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f94113670 0x7f2f94113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39812/0 (socket says 172.21.15.190:39812) 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.500+0000 7f2f993c8640 1 -- 172.21.15.190:0/2491489954 learned_addr learned my addr 172.21.15.190:0/2491489954 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.501+0000 7f2f99bc9640 1 -- 172.21.15.190:0/2491489954 >> v1:172.21.15.97:6789/0 conn(0x7f2f941180e0 legacy=0x7f2f941184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.501+0000 7f2f99bc9640 1 -- 172.21.15.190:0/2491489954 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f94118d90 con 0x7f2f94113670 2024-03-27T08:22:59.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.501+0000 7f2f99bc9640 1 --2- 172.21.15.190:0/2491489954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f94113a80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817a4c9d0290ccbd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.494 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.501+0000 7f2f98bc7640 1 -- 172.21.15.190:0/2491489954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2f84009d90 con 0x7f2f94113670 2024-03-27T08:22:59.494 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.501+0000 7f2f98bc7640 1 -- 172.21.15.190:0/2491489954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2f8400e230 con 0x7f2f94113670 2024-03-27T08:22:59.494 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.501+0000 7f2f98bc7640 1 -- 172.21.15.190:0/2491489954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2f8400e4d0 con 0x7f2f94113670 2024-03-27T08:22:59.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2491489954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 msgr2=0x7f2f94113a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 --2- 172.21.15.190:0/2491489954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f94113a80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2491489954 shutdown_connections 2024-03-27T08:22:59.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 --2- 172.21.15.190:0/2491489954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f94113a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2491489954 >> 172.21.15.190:0/2491489954 conn(0x7f2f9406dab0 msgr2=0x7f2f9406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2491489954 shutdown_connections 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2491489954 wait complete. 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.502+0000 7f2f9be2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f9be2d640 1 Processor -- start 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f9be2d640 1 -- start start 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f9be2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f941aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f9be2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941ae4a0 con 0x7f2f94113670 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f99bc9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f941aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f99bc9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f941aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39820/0 (socket says 172.21.15.190:39820) 2024-03-27T08:22:59.496 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.503+0000 7f2f99bc9640 1 -- 172.21.15.190:0/2418496026 learned_addr learned my addr 172.21.15.190:0/2418496026 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-27T08:22:59.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f99bc9640 1 -- 172.21.15.190:0/2418496026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f941ae5e0 con 0x7f2f94113670 2024-03-27T08:22:59.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f99bc9640 1 --2- 172.21.15.190:0/2418496026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f941aded0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2f84014750 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2f840148b0 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f941ae7d0 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2f84014b50 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f941a8230 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f2f84014d50 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.504+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 977+0+0 (crc 0 0 0) 0x7f2f84019040 con 0x7f2f94113670 2024-03-27T08:22:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.505+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f941ae9c0 con 0x7f2f94113670 2024-03-27T08:22:59.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.507+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f2f84013790 con 0x7f2f94113670 2024-03-27T08:22:59.570 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.578+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b1d0dc27-69b7-42ad-99fb-50b882a4bfa3", "id": 2} v 0) -- 0x7f2f940008d0 con 0x7f2f94113670 2024-03-27T08:22:59.571 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.579+0000 7f2f8a7fc640 1 -- 172.21.15.190:0/2418496026 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1d0dc27-69b7-42ad-99fb-50b882a4bfa3", "id": 2}]=0 v6) ==== 112+0+2 (crc 0 0 0) 0x7f2f840059f0 con 0x7f2f94113670 2024-03-27T08:22:59.571 INFO:teuthology.orchestra.run.smithi190.stdout:2 2024-03-27T08:22:59.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 msgr2=0x7f2f941aded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.572 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 --2- 172.21.15.190:0/2418496026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f941aded0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 shutdown_connections 2024-03-27T08:22:59.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 --2- 172.21.15.190:0/2418496026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f94113670 0x7f2f941aded0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 >> 172.21.15.190:0/2418496026 conn(0x7f2f9406dab0 msgr2=0x7f2f941170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 shutdown_connections 2024-03-27T08:22:59.573 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.580+0000 7f2f9be2d640 1 -- 172.21.15.190:0/2418496026 wait complete. 2024-03-27T08:22:59.583 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new c15879d9-12a6-4558-a5ed-1bc46b9961a2 3 2024-03-27T08:22:59.684 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.692+0000 7f3dfdd70000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T08:22:59.693 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.696+0000 7f1af082d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.694 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.702+0000 7f1af082d640 1 Processor -- start 2024-03-27T08:22:59.694 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.702+0000 7f1af082d640 1 -- start start 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.702+0000 7f1af082d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1aec117340 0x7f1aec117750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1af082d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aec111c60 con 0x7f1aec111140 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1af082d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1aec111df0 con 0x7f1aec117340 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1aea59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1aec117340 0x7f1aec117750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1ae9d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1aec111140 0x7f1aec111550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.190:38328/0 (socket says 172.21.15.190:38328) 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1ae9d9b640 1 -- 172.21.15.190:0/3803471072 learned_addr learned my addr 172.21.15.190:0/3803471072 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1ae959a640 1 -- 172.21.15.190:0/3803471072 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626482209 0 0) 0x7f1aec111c60 con 0x7f1aec111140 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1ae959a640 1 -- 172.21.15.190:0/3803471072 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ad4003580 con 0x7f1aec111140 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1aea59c640 1 -- 172.21.15.190:0/3803471072 >> v1:172.21.15.97:6789/0 conn(0x7f1aec111140 legacy=0x7f1aec111550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.703+0000 7f1aea59c640 1 -- 172.21.15.190:0/3803471072 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec111f60 con 0x7f1aec117340 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.704+0000 7f1aea59c640 1 --2- 172.21.15.190:0/3803471072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec117340 0x7f1aec117750 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b23a5b65651a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.704+0000 7f1ae959a640 1 -- 172.21.15.190:0/3803471072 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1ae0009d90 con 0x7f1aec117340 2024-03-27T08:22:59.696 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.704+0000 7f1ae959a640 1 -- 172.21.15.190:0/3803471072 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1ae000e230 con 0x7f1aec117340 2024-03-27T08:22:59.697 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.704+0000 7f1ae959a640 1 -- 172.21.15.190:0/3803471072 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1ae000e4d0 con 0x7f1aec117340 2024-03-27T08:22:59.697 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.704+0000 7f1af082d640 1 -- 172.21.15.190:0/3803471072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec117340 msgr2=0x7f1aec117750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.697 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.704+0000 7f1af082d640 1 --2- 172.21.15.190:0/3803471072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec117340 0x7f1aec117750 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.697 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.705+0000 7f1af082d640 1 -- 172.21.15.190:0/3803471072 shutdown_connections 2024-03-27T08:22:59.698 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.705+0000 7f1af082d640 1 --2- 172.21.15.190:0/3803471072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec117340 0x7f1aec117750 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.698 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.705+0000 7f1af082d640 1 -- 172.21.15.190:0/3803471072 >> 172.21.15.190:0/3803471072 conn(0x7f1aec070b40 msgr2=0x7f1aec072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.698 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.705+0000 7f1af082d640 1 -- 172.21.15.190:0/3803471072 shutdown_connections 2024-03-27T08:22:59.698 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.705+0000 7f1af082d640 1 -- 172.21.15.190:0/3803471072 wait complete. 2024-03-27T08:22:59.698 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.705+0000 7f1af082d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1af082d640 1 Processor -- start 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1af082d640 1 -- start start 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1af082d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 0x7f1aec1a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1af082d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec1a64f0 con 0x7f1aec111140 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1aea59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 0x7f1aec1a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1aea59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 0x7f1aec1a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.190:39828/0 (socket says 172.21.15.190:39828) 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.706+0000 7f1aea59c640 1 -- 172.21.15.190:0/3711631866 learned_addr learned my addr 172.21.15.190:0/3711631866 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1aea59c640 1 -- 172.21.15.190:0/3711631866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec1a6630 con 0x7f1aec111140 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1aea59c640 1 --2- 172.21.15.190:0/3711631866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 0x7f1aec1a5fb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:22:59.700 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1ae000ee60 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1ae0014620 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1ae00148c0 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aec1a6880 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aec1aa640 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f1ae0013400 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.707+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1092+0+0 (crc 0 0 0) 0x7f1ae0019040 con 0x7f1aec111140 2024-03-27T08:22:59.701 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.708+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aec063450 con 0x7f1aec111140 2024-03-27T08:22:59.702 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.710+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f1ae00124d0 con 0x7f1aec111140 2024-03-27T08:22:59.775 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.783+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c15879d9-12a6-4558-a5ed-1bc46b9961a2", "id": 3} v 0) -- 0x7f1aec1173e0 con 0x7f1aec111140 2024-03-27T08:22:59.779 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.787+0000 7f1adaffd640 1 -- 172.21.15.190:0/3711631866 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c15879d9-12a6-4558-a5ed-1bc46b9961a2", "id": 3}]=0 v7) ==== 112+0+2 (crc 0 0 0) 0x7f1ae0012740 con 0x7f1aec111140 2024-03-27T08:22:59.779 INFO:teuthology.orchestra.run.smithi190.stdout:3 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 msgr2=0x7f1aec1a5fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 --2- 172.21.15.190:0/3711631866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 0x7f1aec1a5fb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 shutdown_connections 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 --2- 172.21.15.190:0/3711631866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aec111140 0x7f1aec1a5fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 >> 172.21.15.190:0/3711631866 conn(0x7f1aec070b40 msgr2=0x7f1aec112e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 shutdown_connections 2024-03-27T08:22:59.780 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-27T08:22:59.788+0000 7f1af082d640 1 -- 172.21.15.190:0/3711631866 wait complete. 2024-03-27T08:22:59.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-03-27T08:22:59.817 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.825+0000 7f3dfdd70000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T08:22:59.827 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-27T08:22:59.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-03-27T08:22:59.871 INFO:tasks.ceph.osd.0:Started 2024-03-27T08:22:59.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-03-27T08:22:59.904 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-27T08:22:59.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-03-27T08:22:59.922 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.930+0000 7f3dfdd70000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T08:22:59.947 INFO:tasks.ceph.osd.1:Started 2024-03-27T08:22:59.947 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-03-27T08:22:59.969 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:22:59.977+0000 7f3dfdd70000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T08:22:59.979 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-27T08:22:59.979 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-03-27T08:23:00.022 INFO:tasks.ceph.osd.2:Started 2024-03-27T08:23:00.022 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-03-27T08:23:00.055 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-27T08:23:00.055 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-03-27T08:23:00.088 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:00.096+0000 7f3dfdd70000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T08:23:00.089 INFO:tasks.ceph.osd.3:Started 2024-03-27T08:23:00.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:23:00.196 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:00.204+0000 7f3dfdd70000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T08:23:00.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.211+0000 7fdb1df5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:00.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.216+0000 7fdb1df5d640 1 Processor -- start 2024-03-27T08:23:00.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.216+0000 7fdb1df5d640 1 -- start start 2024-03-27T08:23:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb1df5d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb18137890 0x7fdb18139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb1df5d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb180735e0 con 0x7fdb18131830 2024-03-27T08:23:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb1df5d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdb180a0260 con 0x7fdb18137890 2024-03-27T08:23:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb16ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb18137890 0x7fdb18139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb177fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdb18131830 0x7fdb18131c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54372/0 (socket says 172.21.15.97:54372) 2024-03-27T08:23:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb177fe640 1 -- 172.21.15.97:0/2893442657 learned_addr learned my addr 172.21.15.97:0/2893442657 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2893442657 >> v1:172.21.15.97:6789/0 conn(0x7fdb18131830 legacy=0x7fdb18131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb167fc640 1 -- 172.21.15.97:0/2893442657 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276597331 0 0) 0x7fdb180735e0 con 0x7fdb18131830 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.217+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2893442657 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1806eb80 con 0x7fdb18137890 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.218+0000 7fdb16ffd640 1 --2- 172.21.15.97:0/2893442657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18137890 0x7fdb18139cb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8ef60f133f2a15 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.218+0000 7fdb167fc640 1 -- 172.21.15.97:0/2893442657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdb0c009d90 con 0x7fdb18137890 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.218+0000 7fdb167fc640 1 -- 172.21.15.97:0/2893442657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdb0c00e230 con 0x7fdb18137890 2024-03-27T08:23:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.218+0000 7fdb167fc640 1 -- 172.21.15.97:0/2893442657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdb0c00e4d0 con 0x7fdb18137890 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 -- 172.21.15.97:0/2893442657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18137890 msgr2=0x7fdb18139cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 --2- 172.21.15.97:0/2893442657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18137890 0x7fdb18139cb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 -- 172.21.15.97:0/2893442657 shutdown_connections 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 --2- 172.21.15.97:0/2893442657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18137890 0x7fdb18139cb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 -- 172.21.15.97:0/2893442657 >> 172.21.15.97:0/2893442657 conn(0x7fdb180a4ae0 msgr2=0x7fdb180a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 -- 172.21.15.97:0/2893442657 shutdown_connections 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 1 -- 172.21.15.97:0/2893442657 wait complete. 2024-03-27T08:23:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.219+0000 7fdb1df5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.220+0000 7fdb1df5d640 1 Processor -- start 2024-03-27T08:23:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.220+0000 7fdb1df5d640 1 -- start start 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.221+0000 7fdb1df5d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 0x7fdb1811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.221+0000 7fdb1df5d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18071f80 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 0x7fdb1811d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 0x7fdb1811d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42154/0 (socket says 172.21.15.97:42154) 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb177fe640 1 -- 172.21.15.97:0/820585718 learned_addr learned my addr 172.21.15.97:0/820585718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb177fe640 1 -- 172.21.15.97:0/820585718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb18071c70 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb177fe640 1 --2- 172.21.15.97:0/820585718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 0x7fdb1811d230 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdb08004b80 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdb08004ce0 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1811fea0 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdb08007120 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1811ea10 con 0x7fdb18131830 2024-03-27T08:23:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fdb0800c540 con 0x7fdb18131830 2024-03-27T08:23:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.222+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7fdb0800fc50 con 0x7fdb18131830 2024-03-27T08:23:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.223+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb18063450 con 0x7fdb18131830 2024-03-27T08:23:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.225+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fdb08007660 con 0x7fdb18131830 2024-03-27T08:23:00.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.286+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb18068a10 con 0x7fdb18131830 2024-03-27T08:23:00.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.287+0000 7fdafffff640 1 -- 172.21.15.97:0/820585718 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7fdb0800f230 con 0x7fdb18131830 2024-03-27T08:23:00.280 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:00.280 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"fsid":"ab3c95de-2422-462d-839c-7edbd2518ef5","created":"2024-03-27T08:22:57.667418+0000","modified":"2024-03-27T08:22:59.784887+0000","last_up_change":"0.000000","last_in_change":"2024-03-27T08:22:59.784887+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"8aee0ca6-e299-4707-bb1a-9d92e6da31c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea23b051-821d-4838-b8ef-1cb78fa81d90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b1d0dc27-69b7-42ad-99fb-50b882a4bfa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c15879d9-12a6-4558-a5ed-1bc46b9961a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:23:00.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 msgr2=0x7fdb1811d230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:00.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 --2- 172.21.15.97:0/820585718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 0x7fdb1811d230 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 shutdown_connections 2024-03-27T08:23:00.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 --2- 172.21.15.97:0/820585718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb18131830 0x7fdb1811d230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 >> 172.21.15.97:0/820585718 conn(0x7fdb180a4ae0 msgr2=0x7fdb1812fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:00.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 shutdown_connections 2024-03-27T08:23:00.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:00.288+0000 7fdb1df5d640 1 -- 172.21.15.97:0/820585718 wait complete. 2024-03-27T08:23:00.294 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-27T08:23:00.294 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-27T08:23:00.369 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:00.377+0000 7f3dfdd70000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T08:23:00.414 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:00.422+0000 7f3dfdd70000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T08:23:00.576 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:00.583+0000 7f3dfdd70000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T08:23:00.697 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-27T08:23:00.801 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.804+0000 7fe5206ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:00.802 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.809+0000 7fe5206ed640 1 Processor -- start 2024-03-27T08:23:00.802 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.809+0000 7fe5206ed640 1 -- start start 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe5206ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5180a1010 0x7fe518134790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe5206ed640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51809f2d0 con 0x7fe518134cd0 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe5206ed640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe51806c9d0 con 0x7fe5180a1010 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51e489640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5180a1010 0x7fe518134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51dc88640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe518134cd0 0x7fe518131300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54386/0 (socket says 172.21.15.97:54386) 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51dc88640 1 -- 172.21.15.97:0/4056036412 learned_addr learned my addr 172.21.15.97:0/4056036412 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51e489640 1 -- 172.21.15.97:0/4056036412 >> v1:172.21.15.97:6789/0 conn(0x7fe518134cd0 legacy=0x7fe518131300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51e489640 1 -- 172.21.15.97:0/4056036412 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51806ea80 con 0x7fe5180a1010 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51e489640 1 --2- 172.21.15.97:0/4056036412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe518134790 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a060428fa13901 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51d487640 1 -- 172.21.15.97:0/4056036412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe50c009d80 con 0x7fe5180a1010 2024-03-27T08:23:00.804 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51d487640 1 -- 172.21.15.97:0/4056036412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe50c00e230 con 0x7fe5180a1010 2024-03-27T08:23:00.805 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.811+0000 7fe51d487640 1 -- 172.21.15.97:0/4056036412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe50c00e4d0 con 0x7fe5180a1010 2024-03-27T08:23:00.805 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.812+0000 7fe5206ed640 1 -- 172.21.15.97:0/4056036412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 msgr2=0x7fe518134790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.812+0000 7fe5206ed640 1 --2- 172.21.15.97:0/4056036412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe518134790 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.812+0000 7fe5206ed640 1 -- 172.21.15.97:0/4056036412 shutdown_connections 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.812+0000 7fe5206ed640 1 --2- 172.21.15.97:0/4056036412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe518134790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.812+0000 7fe5206ed640 1 -- 172.21.15.97:0/4056036412 >> 172.21.15.97:0/4056036412 conn(0x7fe5180a15e0 msgr2=0x7fe5180a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.813+0000 7fe5206ed640 1 -- 172.21.15.97:0/4056036412 shutdown_connections 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.813+0000 7fe5206ed640 1 -- 172.21.15.97:0/4056036412 wait complete. 2024-03-27T08:23:00.806 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.813+0000 7fe5206ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:00.807 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.814+0000 7fe5206ed640 1 Processor -- start 2024-03-27T08:23:00.807 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.814+0000 7fe5206ed640 1 -- start start 2024-03-27T08:23:00.808 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.815+0000 7fe5206ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe5181c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:00.808 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.815+0000 7fe5206ed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe51811f110 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.815+0000 7fe51e489640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe5181c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.815+0000 7fe51e489640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe5181c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42158/0 (socket says 172.21.15.97:42158) 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.815+0000 7fe51e489640 1 -- 172.21.15.97:0/4151811886 learned_addr learned my addr 172.21.15.97:0/4151811886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.815+0000 7fe51e489640 1 -- 172.21.15.97:0/4151811886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe518120440 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe51e489640 1 --2- 172.21.15.97:0/4151811886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe5181c9900 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe50c014540 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe50c0146a0 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe50c014940 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5181208f0 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181c9560 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fe50c013790 con 0x7fe5180a1010 2024-03-27T08:23:00.809 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.816+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7fe50c019040 con 0x7fe5180a1010 2024-03-27T08:23:00.810 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.817+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5181c9750 con 0x7fe5180a1010 2024-03-27T08:23:00.811 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.819+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fe50c012390 con 0x7fe5180a1010 2024-03-27T08:23:00.874 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.880+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe518063450 con 0x7fe5180a1010 2024-03-27T08:23:00.874 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.881+0000 7fe50affd640 1 -- 172.21.15.97:0/4151811886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7fe50c012e60 con 0x7fe5180a1010 2024-03-27T08:23:00.875 INFO:teuthology.misc.health.smithi097.stdout: 2024-03-27T08:23:00.875 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":7,"fsid":"ab3c95de-2422-462d-839c-7edbd2518ef5","created":"2024-03-27T08:22:57.667418+0000","modified":"2024-03-27T08:22:59.784887+0000","last_up_change":"0.000000","last_in_change":"2024-03-27T08:22:59.784887+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"8aee0ca6-e299-4707-bb1a-9d92e6da31c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea23b051-821d-4838-b8ef-1cb78fa81d90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b1d0dc27-69b7-42ad-99fb-50b882a4bfa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c15879d9-12a6-4558-a5ed-1bc46b9961a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 msgr2=0x7fe5181c9900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 --2- 172.21.15.97:0/4151811886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe5181c9900 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 shutdown_connections 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 --2- 172.21.15.97:0/4151811886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5180a1010 0x7fe5181c9900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 >> 172.21.15.97:0/4151811886 conn(0x7fe5180a15e0 msgr2=0x7fe5181bc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 shutdown_connections 2024-03-27T08:23:00.876 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:00.882+0000 7fe5206ed640 1 -- 172.21.15.97:0/4151811886 wait complete. 2024-03-27T08:23:00.884 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-03-27T08:23:01.426 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:01.433+0000 7f3dfdd70000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T08:23:01.704 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:01.712+0000 7f3dfdd70000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T08:23:01.767 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:01.775+0000 7f3dfdd70000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T08:23:01.898 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:01.905+0000 7f3dfdd70000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T08:23:02.924 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:23:02.932+0000 7f3d9ce7d640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-03-27T08:23:07.288 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-27T08:23:07.395 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.399+0000 7f8c9baad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:07.396 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.403+0000 7f8c9baad640 1 Processor -- start 2024-03-27T08:23:07.396 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.404+0000 7f8c9baad640 1 -- start start 2024-03-27T08:23:07.398 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c9baad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c94137930 0x7f8c94139d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:07.398 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c9baad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c940a0c80 con 0x7f8c94131650 2024-03-27T08:23:07.398 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c9baad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f8c940a0eb0 con 0x7f8c94137930 2024-03-27T08:23:07.398 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c99849640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8c94131650 0x7f8c94131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54392/0 (socket says 172.21.15.97:54392) 2024-03-27T08:23:07.398 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c99048640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c94137930 0x7f8c94139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c99849640 1 -- 172.21.15.97:0/2928476082 learned_addr learned my addr 172.21.15.97:0/2928476082 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c98847640 1 -- 172.21.15.97:0/2928476082 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275255734 0 0) 0x7f8c940a0c80 con 0x7f8c94131650 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c98847640 1 -- 172.21.15.97:0/2928476082 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c80003580 con 0x7f8c94131650 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c99048640 1 -- 172.21.15.97:0/2928476082 >> v1:172.21.15.97:6789/0 conn(0x7f8c94131650 legacy=0x7f8c94131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.405+0000 7f8c99048640 1 -- 172.21.15.97:0/2928476082 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c94132170 con 0x7f8c94137930 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.406+0000 7f8c99048640 1 --2- 172.21.15.97:0/2928476082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94137930 0x7f8c94139d50 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8b2b319555d6ef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.406+0000 7f8c98847640 1 -- 172.21.15.97:0/2928476082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c84009d90 con 0x7f8c94137930 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.406+0000 7f8c98847640 1 -- 172.21.15.97:0/2928476082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8c8400e230 con 0x7f8c94137930 2024-03-27T08:23:07.399 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.406+0000 7f8c98847640 1 -- 172.21.15.97:0/2928476082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c8400e4d0 con 0x7f8c94137930 2024-03-27T08:23:07.400 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 -- 172.21.15.97:0/2928476082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94137930 msgr2=0x7f8c94139d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 --2- 172.21.15.97:0/2928476082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94137930 0x7f8c94139d50 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 -- 172.21.15.97:0/2928476082 shutdown_connections 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 --2- 172.21.15.97:0/2928476082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94137930 0x7f8c94139d50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 -- 172.21.15.97:0/2928476082 >> 172.21.15.97:0/2928476082 conn(0x7f8c940a3670 msgr2=0x7f8c940a3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 -- 172.21.15.97:0/2928476082 shutdown_connections 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 1 -- 172.21.15.97:0/2928476082 wait complete. 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.407+0000 7f8c9baad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.408+0000 7f8c9baad640 1 Processor -- start 2024-03-27T08:23:07.401 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.409+0000 7f8c9baad640 1 -- start start 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c9baad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 0x7f8c941c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c9baad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941c5bc0 con 0x7f8c94131650 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c99849640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 0x7f8c941c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c99849640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 0x7f8c941c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42170/0 (socket says 172.21.15.97:42170) 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c99849640 1 -- 172.21.15.97:0/2803287536 learned_addr learned my addr 172.21.15.97:0/2803287536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c99849640 1 -- 172.21.15.97:0/2803287536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c94120420 con 0x7f8c94131650 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c99849640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 0x7f8c941c6170 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:07.403 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c900070b0 con 0x7f8c94131650 2024-03-27T08:23:07.404 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8c90007210 con 0x7f8c94131650 2024-03-27T08:23:07.404 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c941c5e60 con 0x7f8c94131650 2024-03-27T08:23:07.404 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c900074b0 con 0x7f8c94131650 2024-03-27T08:23:07.404 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.410+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c94120820 con 0x7f8c94131650 2024-03-27T08:23:07.405 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.411+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 45686+0+0 (crc 0 0 0) 0x7f8c900076b0 con 0x7f8c94131650 2024-03-27T08:23:07.405 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.411+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c94120620 con 0x7f8c94131650 2024-03-27T08:23:07.405 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.412+0000 7f8c8a7fc640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8c780386c0 0x7f8c7803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:07.405 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.412+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2319+0+0 (crc 0 0 0) 0x7f8c900487b0 con 0x7f8c94131650 2024-03-27T08:23:07.405 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.412+0000 7f8c99048640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8c780386c0 0x7f8c7803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:07.405 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.412+0000 7f8c99048640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8c780386c0 0x7f8c7803abd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:07.409 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.417+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f8c900094a0 con 0x7f8c94131650 2024-03-27T08:23:07.634 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.642+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c94063450 con 0x7f8c94131650 2024-03-27T08:23:07.659 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.643+0000 7f8c8a7fc640 1 -- 172.21.15.97:0/2803287536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+6931 (crc 0 0 0) 0x7f8c900090b0 con 0x7f8c94131650 2024-03-27T08:23:07.660 INFO:teuthology.misc.health.smithi097.stdout: 2024-03-27T08:23:07.660 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":11,"fsid":"ab3c95de-2422-462d-839c-7edbd2518ef5","created":"2024-03-27T08:22:57.667418+0000","modified":"2024-03-27T08:23:06.984853+0000","last_up_change":"2024-03-27T08:23:03.975803+0000","last_in_change":"2024-03-27T08:22:59.784887+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:23:05.955717+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8aee0ca6-e299-4707-bb1a-9d92e6da31c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":3203341735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3203341735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3203341735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3203341735}]},"public_addr":"172.21.15.97:6801/3203341735","cluster_addr":"172.21.15.97:6802/3203341735","heartbeat_back_addr":"172.21.15.97:6812/3203341735","heartbeat_front_addr":"172.21.15.97:6811/3203341735","state":["exists","up"]},{"osd":1,"uuid":"ea23b051-821d-4838-b8ef-1cb78fa81d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":1668842645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1668842645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1668842645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":1668842645}]},"public_addr":"172.21.15.97:6805/1668842645","cluster_addr":"172.21.15.97:6804/1668842645","heartbeat_back_addr":"172.21.15.97:6806/1668842645","heartbeat_front_addr":"172.21.15.97:6803/1668842645","state":["exists","up"]},{"osd":2,"uuid":"b1d0dc27-69b7-42ad-99fb-50b882a4bfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":239686547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":239686547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":239686547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6807","nonce":239686547}]},"public_addr":"172.21.15.190:6809/239686547","cluster_addr":"172.21.15.190:6808/239686547","heartbeat_back_addr":"172.21.15.190:6810/239686547","heartbeat_front_addr":"172.21.15.190:6807/239686547","state":["exists","up"]},{"osd":3,"uuid":"c15879d9-12a6-4558-a5ed-1bc46b9961a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":971408905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":971408905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6803","nonce":971408905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":971408905}]},"public_addr":"172.21.15.190:6805/971408905","cluster_addr":"172.21.15.190:6804/971408905","heartbeat_back_addr":"172.21.15.190:6803/971408905","heartbeat_front_addr":"172.21.15.190:6802/971408905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:23:07.661 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8c780386c0 msgr2=0x7f8c7803abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.661 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8c780386c0 0x7f8c7803abd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.661 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 msgr2=0x7f8c941c6170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.661 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 0x7f8c941c6170 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.661 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 shutdown_connections 2024-03-27T08:23:07.662 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8c780386c0 0x7f8c7803abd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.662 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 --2- 172.21.15.97:0/2803287536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c94131650 0x7f8c941c6170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.662 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.646+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 >> 172.21.15.97:0/2803287536 conn(0x7f8c940a3670 msgr2=0x7f8c94135920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:07.662 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.647+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 shutdown_connections 2024-03-27T08:23:07.662 INFO:teuthology.misc.health.smithi097.stderr:2024-03-27T08:23:07.647+0000 7f8c9baad640 1 -- 172.21.15.97:0/2803287536 wait complete. 2024-03-27T08:23:07.662 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-03-27T08:23:07.662 INFO:tasks.ceph:Creating RBD pool 2024-03-27T08:23:07.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-27T08:23:07.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.776+0000 7fce74fff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:07.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.785+0000 7fce74fff640 1 Processor -- start 2024-03-27T08:23:07.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.785+0000 7fce74fff640 1 -- start start 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce74fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce70137930 0x7fce70139d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce74fff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce7009f320 con 0x7fce7009f710 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce74fff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fce700a1240 con 0x7fce70137930 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce6e59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fce7009f710 0x7fce7009fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54404/0 (socket says 172.21.15.97:54404) 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce6dd9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce70137930 0x7fce70139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce6e59c640 1 -- 172.21.15.97:0/694696282 learned_addr learned my addr 172.21.15.97:0/694696282 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce6dd9b640 1 -- 172.21.15.97:0/694696282 >> v1:172.21.15.97:6789/0 conn(0x7fce7009f710 legacy=0x7fce7009fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce6dd9b640 1 -- 172.21.15.97:0/694696282 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce70073500 con 0x7fce70137930 2024-03-27T08:23:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.786+0000 7fce6dd9b640 1 --2- 172.21.15.97:0/694696282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce70137930 0x7fce70139d50 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f60b086ec4ed6b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.787+0000 7fce6d59a640 1 -- 172.21.15.97:0/694696282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fce60009d90 con 0x7fce70137930 2024-03-27T08:23:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.787+0000 7fce6d59a640 1 -- 172.21.15.97:0/694696282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fce6000e230 con 0x7fce70137930 2024-03-27T08:23:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.787+0000 7fce6d59a640 1 -- 172.21.15.97:0/694696282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fce6000e4d0 con 0x7fce70137930 2024-03-27T08:23:07.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.787+0000 7fce74fff640 1 -- 172.21.15.97:0/694696282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce70137930 msgr2=0x7fce70139d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.787+0000 7fce74fff640 1 --2- 172.21.15.97:0/694696282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce70137930 0x7fce70139d50 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.788+0000 7fce74fff640 1 -- 172.21.15.97:0/694696282 shutdown_connections 2024-03-27T08:23:07.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.788+0000 7fce74fff640 1 --2- 172.21.15.97:0/694696282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce70137930 0x7fce70139d50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.788+0000 7fce74fff640 1 -- 172.21.15.97:0/694696282 >> 172.21.15.97:0/694696282 conn(0x7fce700a15e0 msgr2=0x7fce700a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:07.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.788+0000 7fce74fff640 1 -- 172.21.15.97:0/694696282 shutdown_connections 2024-03-27T08:23:07.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.788+0000 7fce74fff640 1 -- 172.21.15.97:0/694696282 wait complete. 2024-03-27T08:23:07.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.788+0000 7fce74fff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:07.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.789+0000 7fce74fff640 1 Processor -- start 2024-03-27T08:23:07.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.789+0000 7fce74fff640 1 -- start start 2024-03-27T08:23:07.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.790+0000 7fce74fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 0x7fce7011ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:07.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.790+0000 7fce74fff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce70122400 con 0x7fce7009f710 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.790+0000 7fce6e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 0x7fce7011ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.790+0000 7fce6e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 0x7fce7011ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42182/0 (socket says 172.21.15.97:42182) 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.790+0000 7fce6e59c640 1 -- 172.21.15.97:0/3027580763 learned_addr learned my addr 172.21.15.97:0/3027580763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce6e59c640 1 -- 172.21.15.97:0/3027580763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce70121200 con 0x7fce7009f710 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce6e59c640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 0x7fce7011ee80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fce58005a60 con 0x7fce7009f710 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fce58005bc0 con 0x7fce7009f710 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce701220f0 con 0x7fce7009f710 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fce58005e60 con 0x7fce7009f710 2024-03-27T08:23:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.791+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce70121860 con 0x7fce7009f710 2024-03-27T08:23:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.792+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 45686+0+0 (crc 0 0 0) 0x7fce580070b0 con 0x7fce7009f710 2024-03-27T08:23:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.792+0000 7fce66ffd640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fce500386c0 0x7fce5003abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.792+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2319+0+0 (crc 0 0 0) 0x7fce58047cf0 con 0x7fce7009f710 2024-03-27T08:23:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.793+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce700a0200 con 0x7fce7009f710 2024-03-27T08:23:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.793+0000 7fce6dd9b640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fce500386c0 0x7fce5003abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.793+0000 7fce6dd9b640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fce500386c0 0x7fce5003abd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:07.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.798+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fce5801d040 con 0x7fce7009f710 2024-03-27T08:23:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.984+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7fce70121590 con 0x7fce7009f710 2024-03-27T08:23:07.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.991+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fce5800be00 con 0x7fce7009f710 2024-03-27T08:23:07.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.994+0000 7fce66ffd640 1 -- 172.21.15.97:0/3027580763 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) ==== 109+0+0 (crc 0 0 0) 0x7fce5800d330 con 0x7fce7009f710 2024-03-27T08:23:07.986 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'rbd' created 2024-03-27T08:23:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fce500386c0 msgr2=0x7fce5003abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fce500386c0 0x7fce5003abd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 msgr2=0x7fce7011ee80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 0x7fce7011ee80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 shutdown_connections 2024-03-27T08:23:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fce500386c0 0x7fce5003abd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 --2- 172.21.15.97:0/3027580763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce7009f710 0x7fce7011ee80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.996+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 >> 172.21.15.97:0/3027580763 conn(0x7fce700a15e0 msgr2=0x7fce70134530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.997+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 shutdown_connections 2024-03-27T08:23:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:07.997+0000 7fce74fff640 1 -- 172.21.15.97:0/3027580763 wait complete. 2024-03-27T08:23:08.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-27T08:23:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.127+0000 7ff352b1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.137+0000 7ff352b1d640 1 Processor -- start 2024-03-27T08:23:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.138+0000 7ff352b1d640 1 -- start start 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff352b1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff34c137920 0x7ff34c139d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff352b1d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0a3470 con 0x7ff34c09f5f0 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff352b1d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c09f2e0 con 0x7ff34c137920 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff343fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff34c137920 0x7ff34c139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff343fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff34c137920 0x7ff34c139d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42196/0 (socket says 172.21.15.97:42196) 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff343fff640 1 -- 172.21.15.97:0/3185873384 learned_addr learned my addr 172.21.15.97:0/3185873384 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff3508b9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff34c09f5f0 0x7ff34c09fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54410/0 (socket says 172.21.15.97:54410) 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff343fff640 1 -- 172.21.15.97:0/3185873384 >> v1:172.21.15.97:6789/0 conn(0x7ff34c09f5f0 legacy=0x7ff34c09fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff343fff640 1 -- 172.21.15.97:0/3185873384 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c06ea10 con 0x7ff34c137920 2024-03-27T08:23:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff3437fe640 1 -- 172.21.15.97:0/3185873384 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575962513 0 0) 0x7ff34c0a3470 con 0x7ff34c09f5f0 2024-03-27T08:23:08.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff343fff640 1 --2- 172.21.15.97:0/3185873384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c137920 0x7ff34c139d40 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6969c2e897510514 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:08.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff3437fe640 1 -- 172.21.15.97:0/3185873384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff33c009e50 con 0x7ff34c137920 2024-03-27T08:23:08.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff3437fe640 1 -- 172.21.15.97:0/3185873384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff33c00e230 con 0x7ff34c137920 2024-03-27T08:23:08.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.139+0000 7ff3437fe640 1 -- 172.21.15.97:0/3185873384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff33c00e4d0 con 0x7ff34c137920 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 -- 172.21.15.97:0/3185873384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c137920 msgr2=0x7ff34c139d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 --2- 172.21.15.97:0/3185873384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c137920 0x7ff34c139d40 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 -- 172.21.15.97:0/3185873384 shutdown_connections 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 --2- 172.21.15.97:0/3185873384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c137920 0x7ff34c139d40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 -- 172.21.15.97:0/3185873384 >> 172.21.15.97:0/3185873384 conn(0x7ff34c0a15e0 msgr2=0x7ff34c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 -- 172.21.15.97:0/3185873384 shutdown_connections 2024-03-27T08:23:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 1 -- 172.21.15.97:0/3185873384 wait complete. 2024-03-27T08:23:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.140+0000 7ff352b1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.142+0000 7ff352b1d640 1 Processor -- start 2024-03-27T08:23:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.142+0000 7ff352b1d640 1 -- start start 2024-03-27T08:23:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.143+0000 7ff352b1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 0x7ff34c1c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.143+0000 7ff352b1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c121610 con 0x7ff34c09f5f0 2024-03-27T08:23:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.143+0000 7ff3508b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 0x7ff34c1c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.143+0000 7ff3508b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 0x7ff34c1c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42206/0 (socket says 172.21.15.97:42206) 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.143+0000 7ff3508b9640 1 -- 172.21.15.97:0/1737890442 learned_addr learned my addr 172.21.15.97:0/1737890442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.143+0000 7ff3508b9640 1 -- 172.21.15.97:0/1737890442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c120760 con 0x7ff34c09f5f0 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.144+0000 7ff3508b9640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 0x7ff34c1c61f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.144+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff348004b80 con 0x7ff34c09f5f0 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.144+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff348004ce0 con 0x7ff34c09f5f0 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.144+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c11ff70 con 0x7ff34c09f5f0 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.144+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff348007120 con 0x7ff34c09f5f0 2024-03-27T08:23:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.144+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c121e20 con 0x7ff34c09f5f0 2024-03-27T08:23:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.145+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff3480072d0 con 0x7ff34c09f5f0 2024-03-27T08:23:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.145+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c122b40 con 0x7ff34c09f5f0 2024-03-27T08:23:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.145+0000 7ff3417fa640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3380387d0 0x7ff33803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.145+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2661+0+0 (crc 0 0 0) 0x7ff348048030 con 0x7ff34c09f5f0 2024-03-27T08:23:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.145+0000 7ff343fff640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3380387d0 0x7ff33803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.146+0000 7ff343fff640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3380387d0 0x7ff33803ace0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:08.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.151+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff34c122b40 con 0x7ff34c09f5f0 2024-03-27T08:23:08.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:08.345+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7ff34c122b40 con 0x7ff34c09f5f0 2024-03-27T08:23:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.002+0000 7ff3417fa640 1 -- 172.21.15.97:0/1737890442 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v13) ==== 173+0+0 (crc 0 0 0) 0x7ff34c122b40 con 0x7ff34c09f5f0 2024-03-27T08:23:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-27T08:23:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3380387d0 msgr2=0x7ff33803ace0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3380387d0 0x7ff33803ace0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:08.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 msgr2=0x7ff34c1c61f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 0x7ff34c1c61f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 shutdown_connections 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3380387d0 0x7ff33803ace0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 --2- 172.21.15.97:0/1737890442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff34c09f5f0 0x7ff34c1c61f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 >> 172.21.15.97:0/1737890442 conn(0x7ff34c0a15e0 msgr2=0x7ff34c134530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 shutdown_connections 2024-03-27T08:23:08.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.005+0000 7ff352b1d640 1 -- 172.21.15.97:0/1737890442 wait complete. 2024-03-27T08:23:09.013 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-27T08:23:09.014 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-27T08:23:09.014 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-27T08:23:09.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-27T08:23:09.014 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-27T08:23:09.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.143+0000 7fbe9d606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:09.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.148+0000 7fbe9d606640 1 Processor -- start 2024-03-27T08:23:09.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.148+0000 7fbe9d606640 1 -- start start 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe9d606640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe9809cd50 0x7fbe98099470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe9d606640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9806be00 con 0x7fbe980999b0 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe9d606640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9806ea70 con 0x7fbe9809cd50 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe96ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe9809cd50 0x7fbe98099470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe967fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbe980999b0 0x7fbe9809b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47918/0 (socket says 172.21.15.97:47918) 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe96ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe9809cd50 0x7fbe98099470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34590/0 (socket says 172.21.15.97:34590) 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe967fc640 1 -- 172.21.15.97:0/4269720525 learned_addr learned my addr 172.21.15.97:0/4269720525 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe96ffd640 1 -- 172.21.15.97:0/4269720525 >> v1:172.21.15.97:6789/0 conn(0x7fbe980999b0 legacy=0x7fbe9809b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.149+0000 7fbe96ffd640 1 -- 172.21.15.97:0/4269720525 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9809c6e0 con 0x7fbe9809cd50 2024-03-27T08:23:09.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.150+0000 7fbe96ffd640 1 --2- 172.21.15.97:0/4269720525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe9809cd50 0x7fbe98099470 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ef514a83c7fcc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.150+0000 7fbe95ffb640 1 -- 172.21.15.97:0/4269720525 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe8800e040 con 0x7fbe9809cd50 2024-03-27T08:23:09.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.150+0000 7fbe95ffb640 1 -- 172.21.15.97:0/4269720525 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbe88007670 con 0x7fbe9809cd50 2024-03-27T08:23:09.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.150+0000 7fbe95ffb640 1 -- 172.21.15.97:0/4269720525 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe88007910 con 0x7fbe9809cd50 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 -- 172.21.15.97:0/4269720525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe9809cd50 msgr2=0x7fbe98099470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 --2- 172.21.15.97:0/4269720525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe9809cd50 0x7fbe98099470 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 -- 172.21.15.97:0/4269720525 shutdown_connections 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 --2- 172.21.15.97:0/4269720525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe9809cd50 0x7fbe98099470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 -- 172.21.15.97:0/4269720525 >> 172.21.15.97:0/4269720525 conn(0x7fbe980a0e20 msgr2=0x7fbe980a1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 -- 172.21.15.97:0/4269720525 shutdown_connections 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 1 -- 172.21.15.97:0/4269720525 wait complete. 2024-03-27T08:23:09.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.151+0000 7fbe9d606640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:09.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.152+0000 7fbe9d606640 1 Processor -- start 2024-03-27T08:23:09.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.152+0000 7fbe9d606640 1 -- start start 2024-03-27T08:23:09.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.153+0000 7fbe9d606640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 0x7fbe98134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.153+0000 7fbe9d606640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981365f0 con 0x7fbe980999b0 2024-03-27T08:23:09.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.153+0000 7fbe96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 0x7fbe98134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.153+0000 7fbe96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 0x7fbe98134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34606/0 (socket says 172.21.15.97:34606) 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.153+0000 7fbe96ffd640 1 -- 172.21.15.97:0/1950146138 learned_addr learned my addr 172.21.15.97:0/1950146138 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe96ffd640 1 -- 172.21.15.97:0/1950146138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98138680 con 0x7fbe980999b0 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe96ffd640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 0x7fbe98134660 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe8800e040 con 0x7fbe980999b0 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbe88012a90 con 0x7fbe980999b0 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe88012d30 con 0x7fbe980999b0 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe98136a70 con 0x7fbe980999b0 2024-03-27T08:23:09.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.154+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe98135710 con 0x7fbe980999b0 2024-03-27T08:23:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.155+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fbe880153f0 con 0x7fbe980999b0 2024-03-27T08:23:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.155+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe98063450 con 0x7fbe980999b0 2024-03-27T08:23:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.155+0000 7fbe7f7fe640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fbe840387a0 0x7fbe8403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.155+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7fbe880498d0 con 0x7fbe980999b0 2024-03-27T08:23:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.155+0000 7fbe967fc640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fbe840387a0 0x7fbe8403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.156+0000 7fbe967fc640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fbe840387a0 0x7fbe8403acb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.160+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fbe8800fc80 con 0x7fbe980999b0 2024-03-27T08:23:09.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.382+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbe98069290 con 0x7fbe980999b0 2024-03-27T08:23:09.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.384+0000 7fbe7f7fe640 1 -- 172.21.15.97:0/1950146138 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+87605 (crc 0 0 0) 0x7fbe88009210 con 0x7fbe980999b0 2024-03-27T08:23:09.377 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fbe840387a0 msgr2=0x7fbe8403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fbe840387a0 0x7fbe8403acb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 msgr2=0x7fbe98134660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 0x7fbe98134660 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 shutdown_connections 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fbe840387a0 0x7fbe8403acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 --2- 172.21.15.97:0/1950146138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe980999b0 0x7fbe98134660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 >> 172.21.15.97:0/1950146138 conn(0x7fbe980a0e20 msgr2=0x7fbe981313c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 shutdown_connections 2024-03-27T08:23:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.386+0000 7fbe9d606640 1 -- 172.21.15.97:0/1950146138 wait complete. 2024-03-27T08:23:09.389 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":3566337009},{"type":"v1","addr":"172.21.15.190:6801","nonce":3566337009}]},"active_addr":"172.21.15.190:6801/3566337009","active_change":"2024-03-27T08:23:01.920085+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":1813598045}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3015669575}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":1874819910}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2809013672}]}]} 2024-03-27T08:23:09.391 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-27T08:23:09.391 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-27T08:23:09.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:23:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.505+0000 7f76aea64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.509+0000 7f76aea64640 1 Processor -- start 2024-03-27T08:23:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.509+0000 7f76aea64640 1 -- start start 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.510+0000 7f76aea64640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76a81378c0 0x7f76a8139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.510+0000 7f76aea64640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a80a06a0 con 0x7f76a8131650 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.510+0000 7f76aea64640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f76a8132170 con 0x7f76a81378c0 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76a81378c0 0x7f76a8139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76a81378c0 0x7f76a8139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34608/0 (socket says 172.21.15.97:34608) 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a7fff640 1 -- 172.21.15.97:0/1044598034 learned_addr learned my addr 172.21.15.97:0/1044598034 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76ac800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f76a8131650 0x7f76a8131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47926/0 (socket says 172.21.15.97:47926) 2024-03-27T08:23:09.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a7fff640 1 -- 172.21.15.97:0/1044598034 >> v1:172.21.15.97:6789/0 conn(0x7f76a8131650 legacy=0x7f76a8131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a7fff640 1 -- 172.21.15.97:0/1044598034 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a806de30 con 0x7f76a81378c0 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a7fff640 1 --2- 172.21.15.97:0/1044598034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a81378c0 0x7f76a8139ce0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5ec9d568b15844 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a77fe640 1 -- 172.21.15.97:0/1044598034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f769c009e50 con 0x7f76a81378c0 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a77fe640 1 -- 172.21.15.97:0/1044598034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f769c00e230 con 0x7f76a81378c0 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.511+0000 7f76a77fe640 1 -- 172.21.15.97:0/1044598034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f769c00e4d0 con 0x7f76a81378c0 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 -- 172.21.15.97:0/1044598034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a81378c0 msgr2=0x7f76a8139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 --2- 172.21.15.97:0/1044598034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a81378c0 0x7f76a8139ce0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 -- 172.21.15.97:0/1044598034 shutdown_connections 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 --2- 172.21.15.97:0/1044598034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a81378c0 0x7f76a8139ce0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 -- 172.21.15.97:0/1044598034 >> 172.21.15.97:0/1044598034 conn(0x7f76a80a15e0 msgr2=0x7f76a80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 -- 172.21.15.97:0/1044598034 shutdown_connections 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 1 -- 172.21.15.97:0/1044598034 wait complete. 2024-03-27T08:23:09.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.512+0000 7f76aea64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:09.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.514+0000 7f76aea64640 1 Processor -- start 2024-03-27T08:23:09.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.514+0000 7f76aea64640 1 -- start start 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76aea64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 0x7f76a81c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76aea64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8121cc0 con 0x7f76a8131650 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76ac800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 0x7f76a81c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76ac800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 0x7f76a81c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34610/0 (socket says 172.21.15.97:34610) 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76ac800640 1 -- 172.21.15.97:0/476739414 learned_addr learned my addr 172.21.15.97:0/476739414 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76ac800640 1 -- 172.21.15.97:0/476739414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a8120950 con 0x7f76a8131650 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76ac800640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 0x7f76a81c6290 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.515+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f76980071e0 con 0x7f76a8131650 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.516+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7698007340 con 0x7f76a8131650 2024-03-27T08:23:09.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.516+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a81c5be0 con 0x7f76a8131650 2024-03-27T08:23:09.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.516+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f76980075e0 con 0x7f76a8131650 2024-03-27T08:23:09.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.516+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81200f0 con 0x7f76a8131650 2024-03-27T08:23:09.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.517+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a81c6090 con 0x7f76a8131650 2024-03-27T08:23:09.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.517+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f76980077d0 con 0x7f76a8131650 2024-03-27T08:23:09.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.517+0000 7f76a57fa640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f76940387a0 0x7f769403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.517+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7f7698048b10 con 0x7f76a8131650 2024-03-27T08:23:09.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.517+0000 7f76a7fff640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f76940387a0 0x7f769403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.518+0000 7f76a7fff640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f76940387a0 0x7f769403acb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.522+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f7698009470 con 0x7f76a8131650 2024-03-27T08:23:09.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.697+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a8063450 con 0x7f76a8131650 2024-03-27T08:23:09.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.698+0000 7f76a57fa640 1 -- 172.21.15.97:0/476739414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+8723 (crc 0 0 0) 0x7f76980090b0 con 0x7f76a8131650 2024-03-27T08:23:09.690 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:09.690 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":13,"fsid":"ab3c95de-2422-462d-839c-7edbd2518ef5","created":"2024-03-27T08:22:57.667418+0000","modified":"2024-03-27T08:23:08.995342+0000","last_up_change":"2024-03-27T08:23:03.975803+0000","last_in_change":"2024-03-27T08:22:59.784887+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:23:05.955717+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-27T08:23:07.988518+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8aee0ca6-e299-4707-bb1a-9d92e6da31c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":3203341735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3203341735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3203341735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3203341735}]},"public_addr":"172.21.15.97:6801/3203341735","cluster_addr":"172.21.15.97:6802/3203341735","heartbeat_back_addr":"172.21.15.97:6812/3203341735","heartbeat_front_addr":"172.21.15.97:6811/3203341735","state":["exists","up"]},{"osd":1,"uuid":"ea23b051-821d-4838-b8ef-1cb78fa81d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":1668842645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1668842645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1668842645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":1668842645}]},"public_addr":"172.21.15.97:6805/1668842645","cluster_addr":"172.21.15.97:6804/1668842645","heartbeat_back_addr":"172.21.15.97:6806/1668842645","heartbeat_front_addr":"172.21.15.97:6803/1668842645","state":["exists","up"]},{"osd":2,"uuid":"b1d0dc27-69b7-42ad-99fb-50b882a4bfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":239686547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":239686547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":239686547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6807","nonce":239686547}]},"public_addr":"172.21.15.190:6809/239686547","cluster_addr":"172.21.15.190:6808/239686547","heartbeat_back_addr":"172.21.15.190:6810/239686547","heartbeat_front_addr":"172.21.15.190:6807/239686547","state":["exists","up"]},{"osd":3,"uuid":"c15879d9-12a6-4558-a5ed-1bc46b9961a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":971408905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":971408905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6803","nonce":971408905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":971408905}]},"public_addr":"172.21.15.190:6805/971408905","cluster_addr":"172.21.15.190:6804/971408905","heartbeat_back_addr":"172.21.15.190:6803/971408905","heartbeat_front_addr":"172.21.15.190:6802/971408905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:23:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f76940387a0 msgr2=0x7f769403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f76940387a0 0x7f769403acb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 msgr2=0x7f76a81c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 0x7f76a81c6290 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 shutdown_connections 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f76940387a0 0x7f769403acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 --2- 172.21.15.97:0/476739414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a8131650 0x7f76a81c6290 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 >> 172.21.15.97:0/476739414 conn(0x7f76a80a15e0 msgr2=0x7f76a80a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 shutdown_connections 2024-03-27T08:23:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.700+0000 7f76aea64640 1 -- 172.21.15.97:0/476739414 wait complete. 2024-03-27T08:23:09.703 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-27T08:23:09.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:23:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.816+0000 7ffbf82ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:09.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.820+0000 7ffbf82ad640 1 Processor -- start 2024-03-27T08:23:09.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.821+0000 7ffbf82ad640 1 -- start start 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf82ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffbf0131650 0x7ffbf0131a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf82ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbf00a06a0 con 0x7ffbf01345b0 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf82ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ffbf00a03e0 con 0x7ffbf0131650 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf6049640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffbf0131650 0x7ffbf0131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf6049640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffbf0131650 0x7ffbf0131a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34618/0 (socket says 172.21.15.97:34618) 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf6049640 1 -- 172.21.15.97:0/2208652959 learned_addr learned my addr 172.21.15.97:0/2208652959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf5848640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffbf01345b0 0x7ffbf01349c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47940/0 (socket says 172.21.15.97:47940) 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf6049640 1 -- 172.21.15.97:0/2208652959 >> v1:172.21.15.97:6789/0 conn(0x7ffbf01345b0 legacy=0x7ffbf01349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf6049640 1 -- 172.21.15.97:0/2208652959 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf006de30 con 0x7ffbf0131650 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf6049640 1 --2- 172.21.15.97:0/2208652959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0131a60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9f118dc9fca2a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.822+0000 7ffbf5047640 1 -- 172.21.15.97:0/2208652959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffbec00e040 con 0x7ffbf0131650 2024-03-27T08:23:09.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf5047640 1 -- 172.21.15.97:0/2208652959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ffbec0091e0 con 0x7ffbf0131650 2024-03-27T08:23:09.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf5047640 1 -- 172.21.15.97:0/2208652959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffbec007730 con 0x7ffbf0131650 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 -- 172.21.15.97:0/2208652959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 msgr2=0x7ffbf0131a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 --2- 172.21.15.97:0/2208652959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0131a60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 -- 172.21.15.97:0/2208652959 shutdown_connections 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 --2- 172.21.15.97:0/2208652959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0131a60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 -- 172.21.15.97:0/2208652959 >> 172.21.15.97:0/2208652959 conn(0x7ffbf00a15e0 msgr2=0x7ffbf00a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 -- 172.21.15.97:0/2208652959 shutdown_connections 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.823+0000 7ffbf82ad640 1 -- 172.21.15.97:0/2208652959 wait complete. 2024-03-27T08:23:09.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.824+0000 7ffbf82ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:09.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.825+0000 7ffbf82ad640 1 Processor -- start 2024-03-27T08:23:09.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.825+0000 7ffbf82ad640 1 -- start start 2024-03-27T08:23:09.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.826+0000 7ffbf82ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0120000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.826+0000 7ffbf82ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf0131200 con 0x7ffbf0131650 2024-03-27T08:23:09.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.826+0000 7ffbf6049640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0120000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.826+0000 7ffbf6049640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0120000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34630/0 (socket says 172.21.15.97:34630) 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.826+0000 7ffbf6049640 1 -- 172.21.15.97:0/886849030 learned_addr learned my addr 172.21.15.97:0/886849030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbf6049640 1 -- 172.21.15.97:0/886849030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf0120e20 con 0x7ffbf0131650 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbf6049640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0120000 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffbec00e040 con 0x7ffbf0131650 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ffbec007e00 con 0x7ffbf0131650 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf01208a0 con 0x7ffbf0131650 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffbec010960 con 0x7ffbf0131650 2024-03-27T08:23:09.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.827+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf011f870 con 0x7ffbf0131650 2024-03-27T08:23:09.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.828+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ffbec010b10 con 0x7ffbf0131650 2024-03-27T08:23:09.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.828+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf0120c20 con 0x7ffbf0131650 2024-03-27T08:23:09.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.828+0000 7ffbe6ffd640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffbd40387a0 0x7ffbd403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:09.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.828+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7ffbec049410 con 0x7ffbf0131650 2024-03-27T08:23:09.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.828+0000 7ffbf5848640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffbd40387a0 0x7ffbd403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:09.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.829+0000 7ffbf5848640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffbd40387a0 0x7ffbd403acb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:09.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:09.833+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ffbec00e5b0 con 0x7ffbf0131650 2024-03-27T08:23:10.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.013+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbf0063450 con 0x7ffbf0131650 2024-03-27T08:23:10.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.015+0000 7ffbe6ffd640 1 -- 172.21.15.97:0/886849030 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8714 (crc 0 0 0) 0x7ffbec00e1a0 con 0x7ffbf0131650 2024-03-27T08:23:10.008 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:10.008 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":14,"fsid":"ab3c95de-2422-462d-839c-7edbd2518ef5","created":"2024-03-27T08:22:57.667418+0000","modified":"2024-03-27T08:23:10.003628+0000","last_up_change":"2024-03-27T08:23:03.975803+0000","last_in_change":"2024-03-27T08:22:59.784887+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:23:05.955717+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-27T08:23:07.988518+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8aee0ca6-e299-4707-bb1a-9d92e6da31c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":3203341735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3203341735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3203341735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3203341735}]},"public_addr":"172.21.15.97:6801/3203341735","cluster_addr":"172.21.15.97:6802/3203341735","heartbeat_back_addr":"172.21.15.97:6812/3203341735","heartbeat_front_addr":"172.21.15.97:6811/3203341735","state":["exists","up"]},{"osd":1,"uuid":"ea23b051-821d-4838-b8ef-1cb78fa81d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":1668842645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1668842645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1668842645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":1668842645}]},"public_addr":"172.21.15.97:6805/1668842645","cluster_addr":"172.21.15.97:6804/1668842645","heartbeat_back_addr":"172.21.15.97:6806/1668842645","heartbeat_front_addr":"172.21.15.97:6803/1668842645","state":["exists","up"]},{"osd":2,"uuid":"b1d0dc27-69b7-42ad-99fb-50b882a4bfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":239686547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":239686547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":239686547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6807","nonce":239686547}]},"public_addr":"172.21.15.190:6809/239686547","cluster_addr":"172.21.15.190:6808/239686547","heartbeat_back_addr":"172.21.15.190:6810/239686547","heartbeat_front_addr":"172.21.15.190:6807/239686547","state":["exists","up"]},{"osd":3,"uuid":"c15879d9-12a6-4558-a5ed-1bc46b9961a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":971408905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":971408905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6803","nonce":971408905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":971408905}]},"public_addr":"172.21.15.190:6805/971408905","cluster_addr":"172.21.15.190:6804/971408905","heartbeat_back_addr":"172.21.15.190:6803/971408905","heartbeat_front_addr":"172.21.15.190:6802/971408905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:23:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffbd40387a0 msgr2=0x7ffbd403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffbd40387a0 0x7ffbd403acb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 msgr2=0x7ffbf0120000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0120000 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 shutdown_connections 2024-03-27T08:23:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffbd40387a0 0x7ffbd403acb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 --2- 172.21.15.97:0/886849030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbf0131650 0x7ffbf0120000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 >> 172.21.15.97:0/886849030 conn(0x7ffbf00a15e0 msgr2=0x7ffbf01bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 shutdown_connections 2024-03-27T08:23:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.018+0000 7ffbf82ad640 1 -- 172.21.15.97:0/886849030 wait complete. 2024-03-27T08:23:10.021 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-27T08:23:10.021 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-27T08:23:10.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-27T08:23:10.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-27T08:23:10.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.145+0000 7fafdff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.145+0000 7f0a0075d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.145+0000 7ff7ea19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.145+0000 7f22cd8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.150+0000 7fafdff5d640 1 Processor -- start 2024-03-27T08:23:10.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.150+0000 7f0a0075d640 1 Processor -- start 2024-03-27T08:23:10.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.150+0000 7f0a0075d640 1 -- start start 2024-03-27T08:23:10.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.150+0000 7f22cd8ad640 1 Processor -- start 2024-03-27T08:23:10.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.151+0000 7f22cd8ad640 1 -- start start 2024-03-27T08:23:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.151+0000 7ff7ea19d640 1 Processor -- start 2024-03-27T08:23:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.151+0000 7fafdff5d640 1 -- start start 2024-03-27T08:23:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.151+0000 7ff7ea19d640 1 -- start start 2024-03-27T08:23:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdff5d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafd81378c0 0x7fafd8139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f0a0075d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f8131650 0x7f09f8131a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f0a0075d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f80a06a0 con 0x7f09f81345b0 2024-03-27T08:23:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f0a0075d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f09f80a03e0 con 0x7f09f8131650 2024-03-27T08:23:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f09fe4f9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f8131650 0x7f09f8131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdff5d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd80a06a0 con 0x7fafd8131650 2024-03-27T08:23:10.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdff5d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fafd8132170 con 0x7fafd81378c0 2024-03-27T08:23:10.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdd4f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafd81378c0 0x7fafd8139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f09fe4f9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f8131650 0x7f09f8131a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34634/0 (socket says 172.21.15.97:34634) 2024-03-27T08:23:10.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f09fe4f9640 1 -- 172.21.15.97:0/3684951661 learned_addr learned my addr 172.21.15.97:0/3684951661 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdd4f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafd81378c0 0x7fafd8139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34632/0 (socket says 172.21.15.97:34632) 2024-03-27T08:23:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdd4f8640 1 -- 172.21.15.97:0/2802005939 learned_addr learned my addr 172.21.15.97:0/2802005939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f09fe4f9640 1 -- 172.21.15.97:0/3684951661 >> v1:172.21.15.97:6789/0 conn(0x7f09f81345b0 legacy=0x7f09f81349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T08:23:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f09fe4f9640 1 -- 172.21.15.97:0/3684951661 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f806de30 con 0x7f09f8131650 2024-03-27T08:23:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdd4f8640 1 -- 172.21.15.97:0/2802005939 >> v1:172.21.15.97:6789/0 conn(0x7fafd8131650 legacy=0x7fafd8131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T08:23:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdd4f8640 1 -- 172.21.15.97:0/2802005939 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd806de30 con 0x7fafd81378c0 2024-03-27T08:23:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7f09fe4f9640 1 --2- 172.21.15.97:0/3684951661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f8131a60 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaf4f1714274b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.152+0000 7fafdd4f8640 1 --2- 172.21.15.97:0/2802005939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd81378c0 0x7fafd8139ce0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd99c0adfc0d8bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22cd8ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22c8131e50 0x7f22c8136860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22cd8ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c809ce60 con 0x7f22c8131330 2024-03-27T08:23:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22cd8ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f22c809db50 con 0x7f22c8131e50 2024-03-27T08:23:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f09fd4f7640 1 -- 172.21.15.97:0/3684951661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f09f4009dc0 con 0x7f09f8131650 2024-03-27T08:23:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7ea19d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7e412f5e0 0x7ff7e412f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7ea19d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7e4073520 con 0x7ff7e40706b0 2024-03-27T08:23:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22c6ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f22c8131330 0x7f22c8131740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47970/0 (socket says 172.21.15.97:47970) 2024-03-27T08:23:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22c6ffd640 1 -- 172.21.15.97:0/2054882642 learned_addr learned my addr 172.21.15.97:0/2054882642 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7ea19d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e4071880 con 0x7ff7e412f5e0 2024-03-27T08:23:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7fafdccf7640 1 -- 172.21.15.97:0/2802005939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fafc8009dc0 con 0x7fafd81378c0 2024-03-27T08:23:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22c67fc640 1 --2- 172.21.15.97:0/2054882642 >> v2:172.21.15.97:3300/0 conn(0x7f22c8131e50 0x7f22c8136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e2ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff7e40706b0 0x7ff7e4070ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47982/0 (socket says 172.21.15.97:47982) 2024-03-27T08:23:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e2ffd640 1 -- 172.21.15.97:0/2259856509 learned_addr learned my addr 172.21.15.97:0/2259856509 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f09fd4f7640 1 -- 172.21.15.97:0/3684951661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f09f400e230 con 0x7f09f8131650 2024-03-27T08:23:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f09fd4f7640 1 -- 172.21.15.97:0/3684951661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f09f400e4d0 con 0x7f09f8131650 2024-03-27T08:23:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e37fe640 1 --2- 172.21.15.97:0/2259856509 >> v2:172.21.15.97:3300/0 conn(0x7ff7e412f5e0 0x7ff7e412f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22c67fc640 1 -- 172.21.15.97:0/2054882642 >> v1:172.21.15.97:6789/0 conn(0x7f22c8131330 legacy=0x7f22c8131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7f22c67fc640 1 -- 172.21.15.97:0/2054882642 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c809c350 con 0x7f22c8131e50 2024-03-27T08:23:10.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e27fc640 1 -- 172.21.15.97:0/2259856509 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2400784665 0 0) 0x7ff7e4073520 con 0x7ff7e40706b0 2024-03-27T08:23:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7fafdccf7640 1 -- 172.21.15.97:0/2802005939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fafc800e230 con 0x7fafd81378c0 2024-03-27T08:23:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7fafdccf7640 1 -- 172.21.15.97:0/2802005939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fafc800e4d0 con 0x7fafd81378c0 2024-03-27T08:23:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e27fc640 1 -- 172.21.15.97:0/2259856509 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7cc003580 con 0x7ff7e40706b0 2024-03-27T08:23:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e37fe640 1 -- 172.21.15.97:0/2259856509 >> v1:172.21.15.97:6789/0 conn(0x7ff7e40706b0 legacy=0x7ff7e4070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.153+0000 7ff7e37fe640 1 -- 172.21.15.97:0/2259856509 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e4074b40 con 0x7ff7e412f5e0 2024-03-27T08:23:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f22c67fc640 1 --2- 172.21.15.97:0/2054882642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131e50 0x7f22c8136860 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee0cd26f5f8b0c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7ff7e37fe640 1 --2- 172.21.15.97:0/2259856509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e412f5e0 0x7ff7e412f9f0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480945a391521c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7ff7e27fc640 1 -- 172.21.15.97:0/2259856509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7d4009d90 con 0x7ff7e412f5e0 2024-03-27T08:23:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f22c5ffb640 1 -- 172.21.15.97:0/2054882642 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22bc009d90 con 0x7f22c8131e50 2024-03-27T08:23:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7ff7e27fc640 1 -- 172.21.15.97:0/2259856509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff7d400e230 con 0x7ff7e412f5e0 2024-03-27T08:23:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7ff7e27fc640 1 -- 172.21.15.97:0/2259856509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7d4009d90 con 0x7ff7e412f5e0 2024-03-27T08:23:10.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f22c5ffb640 1 -- 172.21.15.97:0/2054882642 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f22bc00e230 con 0x7f22c8131e50 2024-03-27T08:23:10.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f22c5ffb640 1 -- 172.21.15.97:0/2054882642 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22bc00e4d0 con 0x7f22c8131e50 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 -- 172.21.15.97:0/3684951661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 msgr2=0x7f09f8131a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3684951661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f8131a60 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 -- 172.21.15.97:0/3684951661 shutdown_connections 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3684951661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f8131a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 -- 172.21.15.97:0/3684951661 >> 172.21.15.97:0/3684951661 conn(0x7f09f80a15e0 msgr2=0x7f09f80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 -- 172.21.15.97:0/3684951661 shutdown_connections 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 1 -- 172.21.15.97:0/3684951661 wait complete. 2024-03-27T08:23:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.154+0000 7f0a0075d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 -- 172.21.15.97:0/2802005939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd81378c0 msgr2=0x7fafd8139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2802005939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd81378c0 0x7fafd8139ce0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 -- 172.21.15.97:0/2802005939 shutdown_connections 2024-03-27T08:23:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2802005939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd81378c0 0x7fafd8139ce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 -- 172.21.15.97:0/2802005939 >> 172.21.15.97:0/2802005939 conn(0x7fafd80a15e0 msgr2=0x7fafd80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 -- 172.21.15.97:0/2802005939 shutdown_connections 2024-03-27T08:23:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 1 -- 172.21.15.97:0/2802005939 wait complete. 2024-03-27T08:23:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 -- 172.21.15.97:0/2259856509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e412f5e0 msgr2=0x7ff7e412f9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/2259856509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e412f5e0 0x7ff7e412f9f0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7fafdff5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 -- 172.21.15.97:0/2259856509 shutdown_connections 2024-03-27T08:23:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/2259856509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e412f5e0 0x7ff7e412f9f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 -- 172.21.15.97:0/2259856509 >> 172.21.15.97:0/2259856509 conn(0x7ff7e40a2900 msgr2=0x7ff7e40a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 -- 172.21.15.97:0/2259856509 shutdown_connections 2024-03-27T08:23:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7ff7ea19d640 1 -- 172.21.15.97:0/2259856509 wait complete. 2024-03-27T08:23:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7f22cd8ad640 1 -- 172.21.15.97:0/2054882642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131e50 msgr2=0x7f22c8136860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/2054882642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131e50 0x7f22c8136860 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7f22cd8ad640 1 -- 172.21.15.97:0/2054882642 shutdown_connections 2024-03-27T08:23:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.155+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/2054882642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131e50 0x7f22c8136860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7ff7ea19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7f22cd8ad640 1 -- 172.21.15.97:0/2054882642 >> 172.21.15.97:0/2054882642 conn(0x7f22c809e2f0 msgr2=0x7f22c809e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7f22cd8ad640 1 -- 172.21.15.97:0/2054882642 shutdown_connections 2024-03-27T08:23:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7f22cd8ad640 1 -- 172.21.15.97:0/2054882642 wait complete. 2024-03-27T08:23:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7f22cd8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7f0a0075d640 1 Processor -- start 2024-03-27T08:23:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.156+0000 7f0a0075d640 1 -- start start 2024-03-27T08:23:10.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.157+0000 7fafdff5d640 1 Processor -- start 2024-03-27T08:23:10.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.157+0000 7fafdff5d640 1 -- start start 2024-03-27T08:23:10.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7ff7ea19d640 1 Processor -- start 2024-03-27T08:23:10.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7ff7ea19d640 1 -- start start 2024-03-27T08:23:10.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7f0a0075d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f811ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7f0a0075d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8120840 con 0x7f09f8131650 2024-03-27T08:23:10.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7f09fe4f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f811ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7f09fe4f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f811ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34654/0 (socket says 172.21.15.97:34654) 2024-03-27T08:23:10.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7f09fe4f9640 1 -- 172.21.15.97:0/3900285337 learned_addr learned my addr 172.21.15.97:0/3900285337 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.158+0000 7f22cd8ad640 1 Processor -- start 2024-03-27T08:23:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f09fe4f9640 1 -- 172.21.15.97:0/3900285337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f8120ce0 con 0x7f09f8131650 2024-03-27T08:23:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f22cd8ad640 1 -- start start 2024-03-27T08:23:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f09fe4f9640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f811ff30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f09f400ed40 con 0x7f09f8131650 2024-03-27T08:23:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f09f400eea0 con 0x7f09f8131650 2024-03-27T08:23:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f811f810 con 0x7f09f8131650 2024-03-27T08:23:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f09f4015550 con 0x7f09f8131650 2024-03-27T08:23:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7ff7ea19d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 0x7ff7e411deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7ff7ea19d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e411ecf0 con 0x7ff7e40706b0 2024-03-27T08:23:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f811d360 con 0x7f09f8131650 2024-03-27T08:23:10.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7fafdff5d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 0x7fafd811de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7ff7e37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 0x7ff7e411deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7ff7e37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 0x7ff7e411deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34658/0 (socket says 172.21.15.97:34658) 2024-03-27T08:23:10.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.159+0000 7ff7e37fe640 1 -- 172.21.15.97:0/565664746 learned_addr learned my addr 172.21.15.97:0/565664746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7fafdff5d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fafd811eef0 con 0x7fafd8131650 2024-03-27T08:23:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7fafddcf9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 0x7fafd811de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7fafddcf9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 0x7fafd811de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34662/0 (socket says 172.21.15.97:34662) 2024-03-27T08:23:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7fafddcf9640 1 -- 172.21.15.97:0/2139778026 learned_addr learned my addr 172.21.15.97:0/2139778026 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7ff7e37fe640 1 -- 172.21.15.97:0/565664746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e4072800 con 0x7ff7e40706b0 2024-03-27T08:23:10.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f09f4015700 con 0x7f09f8131650 2024-03-27T08:23:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7ff7e37fe640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 0x7ff7e411deb0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09f80a3180 con 0x7f09f8131650 2024-03-27T08:23:10.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7f09eeffd640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f09dc0387a0 0x7f09dc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.160+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7d4013530 con 0x7ff7e40706b0 2024-03-27T08:23:10.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff7d4013690 con 0x7ff7e40706b0 2024-03-27T08:23:10.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f09f4049650 con 0x7f09f8131650 2024-03-27T08:23:10.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e411e750 con 0x7ff7e40706b0 2024-03-27T08:23:10.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7d4013930 con 0x7ff7e40706b0 2024-03-27T08:23:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7fafddcf9640 1 -- 172.21.15.97:0/2139778026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8121840 con 0x7fafd8131650 2024-03-27T08:23:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e411d720 con 0x7ff7e40706b0 2024-03-27T08:23:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22cd8ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 0x7f22c81c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22cd8ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22c81c6150 con 0x7f22c8131330 2024-03-27T08:23:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22c6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 0x7f22c81c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22c6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 0x7f22c81c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34664/0 (socket says 172.21.15.97:34664) 2024-03-27T08:23:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22c6ffd640 1 -- 172.21.15.97:0/960593673 learned_addr learned my addr 172.21.15.97:0/960593673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7fafddcf9640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 0x7fafd811de50 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22c6ffd640 1 -- 172.21.15.97:0/960593673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c811c1c0 con 0x7f22c8131330 2024-03-27T08:23:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fafd400b8b0 con 0x7fafd8131650 2024-03-27T08:23:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fafd400ba10 con 0x7fafd8131650 2024-03-27T08:23:10.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fafd400bcb0 con 0x7fafd8131650 2024-03-27T08:23:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f22c6ffd640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 0x7f22c81c63e0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f09eeffd640 1 --2- 172.21.15.97:0/3900285337 >> v2:172.21.15.97:6805/1668842645 conn(0x7f09dc03d7a0 0x7f09dc03fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 --> v2:172.21.15.97:6805/1668842645 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09dc0402c0 con 0x7f09dc03d7a0 2024-03-27T08:23:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f09f4010a00 con 0x7f09f8131650 2024-03-27T08:23:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.161+0000 7f09fdcf8640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f09dc0387a0 0x7f09dc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7f09fdcf8640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f09dc0387a0 0x7f09dc03acb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7e411eaf0 con 0x7ff7e40706b0 2024-03-27T08:23:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd8120630 con 0x7fafd8131650 2024-03-27T08:23:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22b8005a60 con 0x7f22c8131330 2024-03-27T08:23:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f22b8005bc0 con 0x7f22c8131330 2024-03-27T08:23:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22b8005e60 con 0x7f22c8131330 2024-03-27T08:23:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c8072c70 con 0x7f22c8131330 2024-03-27T08:23:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd8121e60 con 0x7fafd8131650 2024-03-27T08:23:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c811d7e0 con 0x7f22c8131330 2024-03-27T08:23:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.162+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff7d4013a90 con 0x7ff7e40706b0 2024-03-27T08:23:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fafd80a0f20 con 0x7fafd8131650 2024-03-27T08:23:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7ff7cbfff640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff7d0038840 0x7ff7d003ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22c8073120 con 0x7f22c8131330 2024-03-27T08:23:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fafd4010310 con 0x7fafd8131650 2024-03-27T08:23:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff7d4018040 con 0x7ff7e40706b0 2024-03-27T08:23:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7ff7cbfff640 1 --2- 172.21.15.97:0/565664746 >> v2:172.21.15.97:6801/3203341735 conn(0x7ff7d003d840 0x7ff7d003fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 --> v2:172.21.15.97:6801/3203341735 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7d0040360 con 0x7ff7d003d840 2024-03-27T08:23:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7ff7d40123f0 con 0x7ff7e40706b0 2024-03-27T08:23:10.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7fafce7fc640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fafbc0387f0 0x7fafbc03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f22b800c780 con 0x7f22c8131330 2024-03-27T08:23:10.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.163+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fafd404ab10 con 0x7fafd8131650 2024-03-27T08:23:10.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7fafce7fc640 1 --2- 172.21.15.97:0/2139778026 >> v2:172.21.15.190:6805/971408905 conn(0x7fafbc03d7f0 0x7fafbc03fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 --> v2:172.21.15.190:6805/971408905 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafbc040310 con 0x7fafbc03d7f0 2024-03-27T08:23:10.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7fafdd4f8640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fafbc0387f0 0x7fafbc03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7fafd404ad10 con 0x7fafd8131650 2024-03-27T08:23:10.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f22affff640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f22b40387f0 0x7f22b403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f22b8047f40 con 0x7f22c8131330 2024-03-27T08:23:10.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7ff7e2ffd640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff7d0038840 0x7ff7d003ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f22affff640 1 --2- 172.21.15.97:0/960593673 >> v2:172.21.15.190:6809/239686547 conn(0x7f22b403d7f0 0x7f22b403fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 --> v2:172.21.15.190:6809/239686547 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22b4040310 con 0x7f22b403d7f0 2024-03-27T08:23:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f22c67fc640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f22b40387f0 0x7f22b403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f09fecfa640 1 --2- 172.21.15.97:0/3900285337 >> v2:172.21.15.97:6805/1668842645 conn(0x7f09dc03d7a0 0x7f09dc03fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:23:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f22b800d720 con 0x7f22c8131330 2024-03-27T08:23:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7fafdd4f8640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fafbc0387f0 0x7fafbc03ad00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.164+0000 7ff7e2ffd640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff7d0038840 0x7ff7d003ad50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.165+0000 7f22c67fc640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f22b40387f0 0x7f22b403ad00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.165+0000 7fafde4fa640 1 --2- 172.21.15.97:0/2139778026 >> v2:172.21.15.190:6805/971408905 conn(0x7fafbc03d7f0 0x7fafbc03fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:23:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.165+0000 7ff7e3fff640 1 --2- 172.21.15.97:0/565664746 >> v2:172.21.15.97:6801/3203341735 conn(0x7ff7d003d840 0x7ff7d003fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:23:10.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.165+0000 7f22c77fe640 1 --2- 172.21.15.97:0/960593673 >> v2:172.21.15.190:6809/239686547 conn(0x7f22b403d7f0 0x7f22b403fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:23:10.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.168+0000 7fafde4fa640 1 --2- 172.21.15.97:0/2139778026 >> v2:172.21.15.190:6805/971408905 conn(0x7fafbc03d7f0 0x7fafbc03fc80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.169+0000 7f09fecfa640 1 --2- 172.21.15.97:0/3900285337 >> v2:172.21.15.97:6805/1668842645 conn(0x7f09dc03d7a0 0x7f09dc03fc30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.169+0000 7ff7e3fff640 1 --2- 172.21.15.97:0/565664746 >> v2:172.21.15.97:6801/3203341735 conn(0x7ff7d003d840 0x7ff7d003fcd0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.169+0000 7f22c77fe640 1 --2- 172.21.15.97:0/960593673 >> v2:172.21.15.190:6809/239686547 conn(0x7f22b403d7f0 0x7f22b403fc80 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.170+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7fafbc040310 con 0x7fafbc03d7f0 2024-03-27T08:23:10.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.171+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f09dc0402c0 con 0x7f09dc03d7a0 2024-03-27T08:23:10.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.171+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== osd.2 v2:172.21.15.190:6809/239686547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f22b4040310 con 0x7f22b403d7f0 2024-03-27T08:23:10.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.171+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7ff7d0040360 con 0x7ff7d003d840 2024-03-27T08:23:10.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.174+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 --> v2:172.21.15.190:6805/971408905 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fafd80733a0 con 0x7fafbc03d7f0 2024-03-27T08:23:10.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.175+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 --> v2:172.21.15.190:6809/239686547 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22c809ddd0 con 0x7f22b403d7f0 2024-03-27T08:23:10.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.175+0000 7fafce7fc640 1 -- 172.21.15.97:0/2139778026 <== osd.3 v2:172.21.15.190:6805/971408905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fafd000a820 con 0x7fafbc03d7f0 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 >> v2:172.21.15.190:6805/971408905 conn(0x7fafbc03d7f0 msgr2=0x7fafbc03fc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2139778026 >> v2:172.21.15.190:6805/971408905 conn(0x7fafbc03d7f0 0x7fafbc03fc80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fafbc0387f0 msgr2=0x7fafbc03ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fafbc0387f0 0x7fafbc03ad00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 msgr2=0x7fafd811de50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 0x7fafd811de50 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 shutdown_connections 2024-03-27T08:23:10.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2139778026 >> v2:172.21.15.190:6805/971408905 conn(0x7fafbc03d7f0 0x7fafbc03fc80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fafbc0387f0 0x7fafbc03ad00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 --2- 172.21.15.97:0/2139778026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd8131650 0x7fafd811de50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 >> 172.21.15.97:0/2139778026 conn(0x7fafd80a15e0 msgr2=0x7fafd8134530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 shutdown_connections 2024-03-27T08:23:10.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.176+0000 7fafdff5d640 1 -- 172.21.15.97:0/2139778026 wait complete. 2024-03-27T08:23:10.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22affff640 1 -- 172.21.15.97:0/960593673 <== osd.2 v2:172.21.15.190:6809/239686547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f22b4040310 con 0x7f22b403d7f0 2024-03-27T08:23:10.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 >> v2:172.21.15.190:6809/239686547 conn(0x7f22b403d7f0 msgr2=0x7f22b403fc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/960593673 >> v2:172.21.15.190:6809/239686547 conn(0x7f22b403d7f0 0x7f22b403fc80 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f22b40387f0 msgr2=0x7f22b403ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f22b40387f0 0x7f22b403ad00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 msgr2=0x7f22c81c63e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 0x7f22c81c63e0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 shutdown_connections 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/960593673 >> v2:172.21.15.190:6809/239686547 conn(0x7f22b403d7f0 0x7f22b403fc80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f22b40387f0 0x7f22b403ad00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 --2- 172.21.15.97:0/960593673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22c8131330 0x7f22c81c63e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 >> 172.21.15.97:0/960593673 conn(0x7f22c809e2f0 msgr2=0x7f22c8130040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 shutdown_connections 2024-03-27T08:23:10.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.177+0000 7f22cd8ad640 1 -- 172.21.15.97:0/960593673 wait complete. 2024-03-27T08:23:10.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.178+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 --> v2:172.21.15.97:6805/1668842645 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09f80a0f20 con 0x7f09dc03d7a0 2024-03-27T08:23:10.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.178+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 --> v2:172.21.15.97:6801/3203341735 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7e412f830 con 0x7ff7d003d840 2024-03-27T08:23:10.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.181+0000 7f09eeffd640 1 -- 172.21.15.97:0/3900285337 <== osd.1 v2:172.21.15.97:6805/1668842645 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f09f000a820 con 0x7f09dc03d7a0 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.181+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 >> v2:172.21.15.97:6805/1668842645 conn(0x7f09dc03d7a0 msgr2=0x7f09dc03fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.181+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3900285337 >> v2:172.21.15.97:6805/1668842645 conn(0x7f09dc03d7a0 0x7f09dc03fc30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f09dc0387a0 msgr2=0x7f09dc03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f09dc0387a0 0x7f09dc03acb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 msgr2=0x7f09f811ff30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f811ff30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 shutdown_connections 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3900285337 >> v2:172.21.15.97:6805/1668842645 conn(0x7f09dc03d7a0 0x7f09dc03fc30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f09dc0387a0 0x7f09dc03acb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 --2- 172.21.15.97:0/3900285337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f8131650 0x7f09f811ff30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 >> 172.21.15.97:0/3900285337 conn(0x7f09f80a15e0 msgr2=0x7f09f81bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 shutdown_connections 2024-03-27T08:23:10.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.182+0000 7f0a0075d640 1 -- 172.21.15.97:0/3900285337 wait complete. 2024-03-27T08:23:10.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7cbfff640 1 -- 172.21.15.97:0/565664746 <== osd.0 v2:172.21.15.97:6801/3203341735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff7e412f830 con 0x7ff7d003d840 2024-03-27T08:23:10.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 >> v2:172.21.15.97:6801/3203341735 conn(0x7ff7d003d840 msgr2=0x7ff7d003fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/565664746 >> v2:172.21.15.97:6801/3203341735 conn(0x7ff7d003d840 0x7ff7d003fcd0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff7d0038840 msgr2=0x7ff7d003ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff7d0038840 0x7ff7d003ad50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 msgr2=0x7ff7e411deb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 0x7ff7e411deb0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 shutdown_connections 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/565664746 >> v2:172.21.15.97:6801/3203341735 conn(0x7ff7d003d840 0x7ff7d003fcd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff7d0038840 0x7ff7d003ad50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 --2- 172.21.15.97:0/565664746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40706b0 0x7ff7e411deb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.185+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 >> 172.21.15.97:0/565664746 conn(0x7ff7e40a2900 msgr2=0x7ff7e41ba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.186+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 shutdown_connections 2024-03-27T08:23:10.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.186+0000 7ff7ea19d640 1 -- 172.21.15.97:0/565664746 wait complete. 2024-03-27T08:23:10.204 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:10.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-27T08:23:10.205 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:10.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-27T08:23:10.206 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:10.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-27T08:23:10.207 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:10.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-27T08:23:10.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.367+0000 7f390f2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.367+0000 7f96040ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.367+0000 7f3f94660640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.368+0000 7efc6155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7f96040ad640 1 Processor -- start 2024-03-27T08:23:10.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7f390f2ad640 1 Processor -- start 2024-03-27T08:23:10.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7f3f94660640 1 Processor -- start 2024-03-27T08:23:10.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7f3f94660640 1 -- start start 2024-03-27T08:23:10.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7f390f2ad640 1 -- start start 2024-03-27T08:23:10.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7f96040ad640 1 -- start start 2024-03-27T08:23:10.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7efc6155d640 1 Processor -- start 2024-03-27T08:23:10.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.373+0000 7efc6155d640 1 -- start start 2024-03-27T08:23:10.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390f2ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390812e2c0 0x7f390812e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f96040ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95fc09cc10 0x7f95fc135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390f2ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390806e090 con 0x7f3908131320 2024-03-27T08:23:10.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f96040ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc12e2b0 con 0x7f95fc12da20 2024-03-27T08:23:10.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f96040ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc06e4d0 con 0x7f95fc09cc10 2024-03-27T08:23:10.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390f2ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f390809ef70 con 0x7f390812e2c0 2024-03-27T08:23:10.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f3f94660640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f8c1378c0 0x7f3f8c139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390d049640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390812e2c0 0x7f390812e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f3f94660640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f8c0a06a0 con 0x7f3f8c131650 2024-03-27T08:23:10.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f3f94660640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8c132170 con 0x7f3f8c1378c0 2024-03-27T08:23:10.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f9601648640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95fc09cc10 0x7f95fc135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390c848640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3908131320 0x7f3908131730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47990/0 (socket says 172.21.15.97:47990) 2024-03-27T08:23:10.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390c848640 1 -- 172.21.15.97:0/3435360679 learned_addr learned my addr 172.21.15.97:0/3435360679 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f3f923fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3f8c131650 0x7f3f8c131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47992/0 (socket says 172.21.15.97:47992) 2024-03-27T08:23:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f3f923fc640 1 -- 172.21.15.97:0/1291034073 learned_addr learned my addr 172.21.15.97:0/1291034073 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f390d049640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390812e2c0 0x7f390812e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34666/0 (socket says 172.21.15.97:34666) 2024-03-27T08:23:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f9601648640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95fc09cc10 0x7f95fc135cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34674/0 (socket says 172.21.15.97:34674) 2024-03-27T08:23:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.374+0000 7f9601648640 1 -- 172.21.15.97:0/3680732749 learned_addr learned my addr 172.21.15.97:0/3680732749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc6155d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efc5c131330 0x7efc5c131740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f3f91bfb640 1 --2- 172.21.15.97:0/1291034073 >> v2:172.21.15.97:3300/0 conn(0x7f3f8c1378c0 0x7f3f8c139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc6155d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc5c09ce60 con 0x7efc5c131c80 2024-03-27T08:23:10.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc6155d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7efc5c09db50 con 0x7efc5c131330 2024-03-27T08:23:10.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc5affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efc5c131330 0x7efc5c131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc5affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efc5c131330 0x7efc5c131740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34692/0 (socket says 172.21.15.97:34692) 2024-03-27T08:23:10.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc5affd640 1 -- 172.21.15.97:0/892371545 learned_addr learned my addr 172.21.15.97:0/892371545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f38fffff640 1 -- 172.21.15.97:0/3435360679 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2122696171 0 0) 0x7f390806e090 con 0x7f3908131320 2024-03-27T08:23:10.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f38fffff640 1 -- 172.21.15.97:0/3435360679 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38f4003580 con 0x7f3908131320 2024-03-27T08:23:10.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f9601648640 1 -- 172.21.15.97:0/3680732749 >> v1:172.21.15.97:6789/0 conn(0x7f95fc12da20 legacy=0x7f95fc09c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f9601648640 1 -- 172.21.15.97:0/3680732749 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc12dfa0 con 0x7f95fc09cc10 2024-03-27T08:23:10.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f390d049640 1 -- 172.21.15.97:0/3435360679 >> v1:172.21.15.97:6789/0 conn(0x7f3908131320 legacy=0x7f3908131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f390d049640 1 -- 172.21.15.97:0/3435360679 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39080734e0 con 0x7f390812e2c0 2024-03-27T08:23:10.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f9601648640 1 --2- 172.21.15.97:0/3680732749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc135cc0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa63772a26bdac1b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc5affd640 1 -- 172.21.15.97:0/892371545 >> v1:172.21.15.97:6789/0 conn(0x7efc5c131c80 legacy=0x7efc5c12e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f3f913fa640 1 -- 172.21.15.97:0/1291034073 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3074628703 0 0) 0x7f3f8c0a06a0 con 0x7f3f8c131650 2024-03-27T08:23:10.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f3f913fa640 1 -- 172.21.15.97:0/1291034073 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f78003580 con 0x7f3f8c131650 2024-03-27T08:23:10.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f3f91bfb640 1 -- 172.21.15.97:0/1291034073 >> v1:172.21.15.97:6789/0 conn(0x7f3f8c131650 legacy=0x7f3f8c131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f3f91bfb640 1 -- 172.21.15.97:0/1291034073 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c06de30 con 0x7f3f8c1378c0 2024-03-27T08:23:10.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7f390d049640 1 --2- 172.21.15.97:0/3435360679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390812e6d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82a5e06ca5f3b10 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.375+0000 7efc5affd640 1 -- 172.21.15.97:0/892371545 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c09c350 con 0x7efc5c131330 2024-03-27T08:23:10.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7efc5affd640 1 --2- 172.21.15.97:0/892371545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c131740 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cf00090edb5a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f38fffff640 1 -- 172.21.15.97:0/3435360679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3904010570 con 0x7f390812e2c0 2024-03-27T08:23:10.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f38fffff640 1 -- 172.21.15.97:0/3435360679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f39040106d0 con 0x7f390812e2c0 2024-03-27T08:23:10.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f38fffff640 1 -- 172.21.15.97:0/3435360679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3904010570 con 0x7f390812e2c0 2024-03-27T08:23:10.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7efc59ffb640 1 -- 172.21.15.97:0/892371545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc4c009e50 con 0x7efc5c131330 2024-03-27T08:23:10.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f9600e47640 1 -- 172.21.15.97:0/3680732749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95ec009e50 con 0x7f95fc09cc10 2024-03-27T08:23:10.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f9600e47640 1 -- 172.21.15.97:0/3680732749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f95ec00e230 con 0x7f95fc09cc10 2024-03-27T08:23:10.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f9600e47640 1 -- 172.21.15.97:0/3680732749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95ec00e4d0 con 0x7f95fc09cc10 2024-03-27T08:23:10.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7efc59ffb640 1 -- 172.21.15.97:0/892371545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc4c00e230 con 0x7efc5c131330 2024-03-27T08:23:10.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f3f91bfb640 1 --2- 172.21.15.97:0/1291034073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c1378c0 0x7f3f8c139ce0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2381593e740f6f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7efc59ffb640 1 -- 172.21.15.97:0/892371545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc4c00e4d0 con 0x7efc5c131330 2024-03-27T08:23:10.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f3f913fa640 1 -- 172.21.15.97:0/1291034073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3f7c009d90 con 0x7f3f8c1378c0 2024-03-27T08:23:10.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f3f913fa640 1 -- 172.21.15.97:0/1291034073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3f7c00e230 con 0x7f3f8c1378c0 2024-03-27T08:23:10.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.376+0000 7f3f913fa640 1 -- 172.21.15.97:0/1291034073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3f7c009d90 con 0x7f3f8c1378c0 2024-03-27T08:23:10.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 -- 172.21.15.97:0/3435360679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 msgr2=0x7f390812e6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 --2- 172.21.15.97:0/3435360679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390812e6d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 -- 172.21.15.97:0/3435360679 shutdown_connections 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 --2- 172.21.15.97:0/3435360679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390812e6d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 -- 172.21.15.97:0/3435360679 >> 172.21.15.97:0/3435360679 conn(0x7f390809d1d0 msgr2=0x7f390809d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 -- 172.21.15.97:0/3435360679 shutdown_connections 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 -- 172.21.15.97:0/892371545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 msgr2=0x7efc5c131740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 --2- 172.21.15.97:0/892371545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c131740 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 -- 172.21.15.97:0/3680732749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 msgr2=0x7f95fc135cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 --2- 172.21.15.97:0/3680732749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc135cc0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 1 -- 172.21.15.97:0/3435360679 wait complete. 2024-03-27T08:23:10.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 -- 172.21.15.97:0/892371545 shutdown_connections 2024-03-27T08:23:10.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 --2- 172.21.15.97:0/892371545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c131740 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 -- 172.21.15.97:0/892371545 >> 172.21.15.97:0/892371545 conn(0x7efc5c09e2f0 msgr2=0x7efc5c09e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 -- 172.21.15.97:0/892371545 shutdown_connections 2024-03-27T08:23:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 -- 172.21.15.97:0/3680732749 shutdown_connections 2024-03-27T08:23:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 --2- 172.21.15.97:0/3680732749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc135cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 -- 172.21.15.97:0/3680732749 >> 172.21.15.97:0/3680732749 conn(0x7f95fc09d1d0 msgr2=0x7f95fc09d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 -- 172.21.15.97:0/1291034073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c1378c0 msgr2=0x7f3f8c139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 --2- 172.21.15.97:0/1291034073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c1378c0 0x7f3f8c139ce0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 1 -- 172.21.15.97:0/892371545 wait complete. 2024-03-27T08:23:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 -- 172.21.15.97:0/3680732749 shutdown_connections 2024-03-27T08:23:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f390f2ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 1 -- 172.21.15.97:0/3680732749 wait complete. 2024-03-27T08:23:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 -- 172.21.15.97:0/1291034073 shutdown_connections 2024-03-27T08:23:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 --2- 172.21.15.97:0/1291034073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c1378c0 0x7f3f8c139ce0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 -- 172.21.15.97:0/1291034073 >> 172.21.15.97:0/1291034073 conn(0x7f3f8c0a15e0 msgr2=0x7f3f8c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 -- 172.21.15.97:0/1291034073 shutdown_connections 2024-03-27T08:23:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 1 -- 172.21.15.97:0/1291034073 wait complete. 2024-03-27T08:23:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7efc6155d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f96040ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.377+0000 7f3f94660640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.378+0000 7f390f2ad640 1 Processor -- start 2024-03-27T08:23:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7f390f2ad640 1 -- start start 2024-03-27T08:23:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7f3f94660640 1 Processor -- start 2024-03-27T08:23:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7f3f94660640 1 -- start start 2024-03-27T08:23:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7f96040ad640 1 Processor -- start 2024-03-27T08:23:10.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7efc6155d640 1 Processor -- start 2024-03-27T08:23:10.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7f96040ad640 1 -- start start 2024-03-27T08:23:10.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.379+0000 7efc6155d640 1 -- start start 2024-03-27T08:23:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f390f2ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390811bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f390f2ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f390812db40 con 0x7f390812e2c0 2024-03-27T08:23:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f390d049640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390811bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f3f94660640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 0x7f3f8c11e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f3f94660640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c11df30 con 0x7f3f8c131650 2024-03-27T08:23:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f390d049640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390811bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34694/0 (socket says 172.21.15.97:34694) 2024-03-27T08:23:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f390d049640 1 -- 172.21.15.97:0/859604582 learned_addr learned my addr 172.21.15.97:0/859604582 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f96040ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f96040ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc11c070 con 0x7f95fc09cc10 2024-03-27T08:23:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f3f923fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 0x7f3f8c11e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f3f923fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 0x7f3f8c11e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34698/0 (socket says 172.21.15.97:34698) 2024-03-27T08:23:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f3f923fc640 1 -- 172.21.15.97:0/772774608 learned_addr learned my addr 172.21.15.97:0/772774608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f9601e49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f9601e49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34712/0 (socket says 172.21.15.97:34712) 2024-03-27T08:23:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f9601e49640 1 -- 172.21.15.97:0/2551905147 learned_addr learned my addr 172.21.15.97:0/2551905147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7efc6155d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c1c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7efc6155d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c13a3c0 con 0x7efc5c131330 2024-03-27T08:23:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7efc5affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c1c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7f3f923fc640 1 -- 172.21.15.97:0/772774608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c11ff30 con 0x7f3f8c131650 2024-03-27T08:23:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7efc5affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c1c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34724/0 (socket says 172.21.15.97:34724) 2024-03-27T08:23:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.380+0000 7efc5affd640 1 -- 172.21.15.97:0/2574682759 learned_addr learned my addr 172.21.15.97:0/2574682759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f3f923fc640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 0x7f3f8c11e3f0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3f88005a60 con 0x7f3f8c131650 2024-03-27T08:23:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f390d049640 1 -- 172.21.15.97:0/859604582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390812ddc0 con 0x7f390812e2c0 2024-03-27T08:23:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c11db90 con 0x7f3f8c131650 2024-03-27T08:23:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3f88005bc0 con 0x7f3f8c131650 2024-03-27T08:23:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f9601e49640 1 -- 172.21.15.97:0/2551905147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc119c10 con 0x7f95fc09cc10 2024-03-27T08:23:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3f88005e60 con 0x7f3f8c131650 2024-03-27T08:23:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c1215a0 con 0x7f3f8c131650 2024-03-27T08:23:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7efc5affd640 1 -- 172.21.15.97:0/2574682759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c071510 con 0x7efc5c131330 2024-03-27T08:23:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f9601e49640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc119fa0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f390d049640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390811bab0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7efc5affd640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c1c9840 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3904010570 con 0x7f390812e2c0 2024-03-27T08:23:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95f800c870 con 0x7f95fc09cc10 2024-03-27T08:23:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f390f2ad640 1 -- 172.21.15.97:0/859604582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390811f120 con 0x7f390812e2c0 2024-03-27T08:23:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc11bac0 con 0x7f95fc09cc10 2024-03-27T08:23:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f95f800c9d0 con 0x7f95fc09cc10 2024-03-27T08:23:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.381+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95f800cc70 con 0x7f95fc09cc10 2024-03-27T08:23:10.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc11d660 con 0x7f95fc09cc10 2024-03-27T08:23:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3904016040 con 0x7f390812e2c0 2024-03-27T08:23:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7f390f2ad640 1 -- 172.21.15.97:0/859604582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390806a9e0 con 0x7f390812e2c0 2024-03-27T08:23:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f3f880070b0 con 0x7f3f8c131650 2024-03-27T08:23:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f3f82ffd640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3f700387f0 0x7f3f7003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f3f88048020 con 0x7f3f8c131650 2024-03-27T08:23:10.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f390400fca0 con 0x7f390812e2c0 2024-03-27T08:23:10.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e00050f0 con 0x7f390812e2c0 2024-03-27T08:23:10.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c1310d0 con 0x7f3f8c131650 2024-03-27T08:23:10.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f95f8006830 con 0x7f95fc09cc10 2024-03-27T08:23:10.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f95f27fc640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95e00387f0 0x7f95e003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.383+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f95f80483e0 con 0x7f95fc09cc10 2024-03-27T08:23:10.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc4c00ede0 con 0x7efc5c131330 2024-03-27T08:23:10.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc4c015420 con 0x7efc5c131330 2024-03-27T08:23:10.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f3f91bfb640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3f700387f0 0x7f3f7003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc4c0156c0 con 0x7efc5c131330 2024-03-27T08:23:10.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c073030 con 0x7efc5c131330 2024-03-27T08:23:10.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.382+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c071ab0 con 0x7efc5c131330 2024-03-27T08:23:10.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc11be70 con 0x7f95fc09cc10 2024-03-27T08:23:10.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f390400e360 con 0x7f390812e2c0 2024-03-27T08:23:10.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7efc4c015820 con 0x7efc5c131330 2024-03-27T08:23:10.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c0050f0 con 0x7efc5c131330 2024-03-27T08:23:10.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f38fdffb640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38ec038750 0x7f38ec03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f390404bc60 con 0x7f390812e2c0 2024-03-27T08:23:10.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f9601648640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95e00387f0 0x7f95e003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7efc437fe640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7efc480387f0 0x7efc4803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:10.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f3f91bfb640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3f700387f0 0x7f3f7003ad00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7efc4c0497c0 con 0x7efc5c131330 2024-03-27T08:23:10.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7efc5a7fc640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7efc480387f0 0x7efc4803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.384+0000 7f390c848640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38ec038750 0x7f38ec03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:10.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.385+0000 7f9601648640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95e00387f0 0x7f95e003ad00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.385+0000 7efc5a7fc640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7efc480387f0 0x7efc4803ad00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.385+0000 7f390c848640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38ec038750 0x7f38ec03ac60 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.389+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f3f8800da20 con 0x7f3f8c131650 2024-03-27T08:23:10.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.399+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f95f800adc0 con 0x7f95fc09cc10 2024-03-27T08:23:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.410+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7efc4c012840 con 0x7efc5c131330 2024-03-27T08:23:10.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.420+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f390400fea0 con 0x7f390812e2c0 2024-03-27T08:23:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.590+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3f8c063450 con 0x7f3f8c131650 2024-03-27T08:23:10.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.591+0000 7f3f82ffd640 1 -- 172.21.15.97:0/772774608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f3f8800d610 con 0x7f3f8c131650 2024-03-27T08:23:10.583 INFO:teuthology.orchestra.run.smithi097.stdout:38654705669 2024-03-27T08:23:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.592+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f95fc063450 con 0x7f95fc09cc10 2024-03-27T08:23:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.592+0000 7f95f27fc640 1 -- 172.21.15.97:0/2551905147 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f95f800e1c0 con 0x7f95fc09cc10 2024-03-27T08:23:10.585 INFO:teuthology.orchestra.run.smithi097.stdout:38654705669 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3f700387f0 msgr2=0x7f3f7003ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3f700387f0 0x7f3f7003ad00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 msgr2=0x7f3f8c11e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 0x7f3f8c11e3f0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 shutdown_connections 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3f700387f0 0x7f3f7003ad00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 --2- 172.21.15.97:0/772774608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f8c131650 0x7f3f8c11e3f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 >> 172.21.15.97:0/772774608 conn(0x7f3f8c0a15e0 msgr2=0x7f3f8c134530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 shutdown_connections 2024-03-27T08:23:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.594+0000 7f3f94660640 1 -- 172.21.15.97:0/772774608 wait complete. 2024-03-27T08:23:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95e00387f0 msgr2=0x7f95e003ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95e00387f0 0x7f95e003ad00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 msgr2=0x7f95fc119fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc119fa0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 shutdown_connections 2024-03-27T08:23:10.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95e00387f0 0x7f95e003ad00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 --2- 172.21.15.97:0/2551905147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95fc09cc10 0x7f95fc119fa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 >> 172.21.15.97:0/2551905147 conn(0x7f95fc09d1d0 msgr2=0x7f95fc09bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 shutdown_connections 2024-03-27T08:23:10.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.595+0000 7f96040ad640 1 -- 172.21.15.97:0/2551905147 wait complete. 2024-03-27T08:23:10.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.598+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efc3c0053a0 con 0x7efc5c131330 2024-03-27T08:23:10.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.598+0000 7efc437fe640 1 -- 172.21.15.97:0/2574682759 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7efc4c012840 con 0x7efc5c131330 2024-03-27T08:23:10.590 INFO:teuthology.orchestra.run.smithi097.stdout:38654705669 2024-03-27T08:23:10.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7efc480387f0 msgr2=0x7efc4803ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7efc480387f0 0x7efc4803ad00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 msgr2=0x7efc5c1c9840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c1c9840 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 shutdown_connections 2024-03-27T08:23:10.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7efc480387f0 0x7efc4803ad00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 --2- 172.21.15.97:0/2574682759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c131330 0x7efc5c1c9840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 >> 172.21.15.97:0/2574682759 conn(0x7efc5c09e2f0 msgr2=0x7efc5c12a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 shutdown_connections 2024-03-27T08:23:10.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.601+0000 7efc6155d640 1 -- 172.21.15.97:0/2574682759 wait complete. 2024-03-27T08:23:10.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-03-27T08:23:10.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-03-27T08:23:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.612+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f38e00053a0 con 0x7f390812e2c0 2024-03-27T08:23:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.612+0000 7f38fdffb640 1 -- 172.21.15.97:0/859604582 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f390400c2e0 con 0x7f390812e2c0 2024-03-27T08:23:10.605 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:10.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38ec038750 msgr2=0x7f38ec03ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38ec038750 0x7f38ec03ac60 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 msgr2=0x7f390811bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390811bab0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 shutdown_connections 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38ec038750 0x7f38ec03ac60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 --2- 172.21.15.97:0/859604582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390812e2c0 0x7f390811bab0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.614+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 >> 172.21.15.97:0/859604582 conn(0x7f390809d1d0 msgr2=0x7f390812fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.615+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 shutdown_connections 2024-03-27T08:23:10.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:10.615+0000 7f38f37fe640 1 -- 172.21.15.97:0/859604582 wait complete. 2024-03-27T08:23:10.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-03-27T08:23:10.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.3 2024-03-27T08:23:10.617 DEBUG:teuthology.parallel:result is None 2024-03-27T08:23:11.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-27T08:23:11.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-27T08:23:11.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-27T08:23:11.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.726+0000 7f44e2f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.726+0000 7f0cfc35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.732+0000 7f0cfc35d640 1 Processor -- start 2024-03-27T08:23:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.732+0000 7f0cfc35d640 1 -- start start 2024-03-27T08:23:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.732+0000 7f44e2f1d640 1 Processor -- start 2024-03-27T08:23:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.732+0000 7f44e2f1d640 1 -- start start 2024-03-27T08:23:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f44e2f1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f44dc131830 0x7f44dc131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f44e2f1d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44dc0735e0 con 0x7f44dc134580 2024-03-27T08:23:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f44e2f1d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f44dc0a0260 con 0x7f44dc131830 2024-03-27T08:23:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f0cfc35d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cf4137b30 0x7f0cf4139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f0cfc35d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf406fd00 con 0x7f0cf4131880 2024-03-27T08:23:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f0cfc35d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf4071010 con 0x7f0cf4137b30 2024-03-27T08:23:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f0cf98f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cf4137b30 0x7f0cf4139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f44e0cb9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f44dc131830 0x7f44dc131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f44e0cb9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f44dc131830 0x7f44dc131c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34732/0 (socket says 172.21.15.97:34732) 2024-03-27T08:23:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f44e0cb9640 1 -- 172.21.15.97:0/3143267934 learned_addr learned my addr 172.21.15.97:0/3143267934 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f0cf98f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cf4137b30 0x7f0cf4139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34748/0 (socket says 172.21.15.97:34748) 2024-03-27T08:23:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.733+0000 7f0cf98f8640 1 -- 172.21.15.97:0/2434834700 learned_addr learned my addr 172.21.15.97:0/2434834700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f44e0cb9640 1 -- 172.21.15.97:0/3143267934 >> v1:172.21.15.97:6789/0 conn(0x7f44dc134580 legacy=0x7f44dc134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f44e0cb9640 1 -- 172.21.15.97:0/3143267934 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc06eb80 con 0x7f44dc131830 2024-03-27T08:23:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f0cf98f8640 1 -- 172.21.15.97:0/2434834700 >> v1:172.21.15.97:6789/0 conn(0x7f0cf4131880 legacy=0x7f0cf4131c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T08:23:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f0cf98f8640 1 -- 172.21.15.97:0/2434834700 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf4070510 con 0x7f0cf4137b30 2024-03-27T08:23:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f44e0cb9640 1 --2- 172.21.15.97:0/3143267934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc131c40 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53de24c6eb09d326 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f0cf98f8640 1 --2- 172.21.15.97:0/2434834700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4137b30 0x7f0cf4139f50 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1197a553b0ea5a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f44db7fe640 1 -- 172.21.15.97:0/3143267934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f44d0009e50 con 0x7f44dc131830 2024-03-27T08:23:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f44db7fe640 1 -- 172.21.15.97:0/3143267934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f44d000e230 con 0x7f44dc131830 2024-03-27T08:23:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f44db7fe640 1 -- 172.21.15.97:0/3143267934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f44d000e4d0 con 0x7f44dc131830 2024-03-27T08:23:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f0cf90f7640 1 -- 172.21.15.97:0/2434834700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ce4009dc0 con 0x7f0cf4137b30 2024-03-27T08:23:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f0cf90f7640 1 -- 172.21.15.97:0/2434834700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0ce400e230 con 0x7f0cf4137b30 2024-03-27T08:23:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.734+0000 7f0cf90f7640 1 -- 172.21.15.97:0/2434834700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ce400e4d0 con 0x7f0cf4137b30 2024-03-27T08:23:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 -- 172.21.15.97:0/3143267934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 msgr2=0x7f44dc131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 -- 172.21.15.97:0/2434834700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4137b30 msgr2=0x7f0cf4139f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 --2- 172.21.15.97:0/3143267934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc131c40 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 --2- 172.21.15.97:0/2434834700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4137b30 0x7f0cf4139f50 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 -- 172.21.15.97:0/3143267934 shutdown_connections 2024-03-27T08:23:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 --2- 172.21.15.97:0/3143267934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc131c40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 -- 172.21.15.97:0/3143267934 >> 172.21.15.97:0/3143267934 conn(0x7f44dc0a4ae0 msgr2=0x7f44dc0a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 -- 172.21.15.97:0/2434834700 shutdown_connections 2024-03-27T08:23:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 --2- 172.21.15.97:0/2434834700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4137b30 0x7f0cf4139f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 -- 172.21.15.97:0/2434834700 >> 172.21.15.97:0/2434834700 conn(0x7f0cf40a2900 msgr2=0x7f0cf40a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 -- 172.21.15.97:0/3143267934 shutdown_connections 2024-03-27T08:23:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f44e2f1d640 1 -- 172.21.15.97:0/3143267934 wait complete. 2024-03-27T08:23:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 -- 172.21.15.97:0/2434834700 shutdown_connections 2024-03-27T08:23:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.735+0000 7f0cfc35d640 1 -- 172.21.15.97:0/2434834700 wait complete. 2024-03-27T08:23:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.736+0000 7f44e2f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.736+0000 7f0cfc35d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.737+0000 7f44e2f1d640 1 Processor -- start 2024-03-27T08:23:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.737+0000 7f44e2f1d640 1 -- start start 2024-03-27T08:23:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.738+0000 7f0cfc35d640 1 Processor -- start 2024-03-27T08:23:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.738+0000 7f0cfc35d640 1 -- start start 2024-03-27T08:23:11.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.738+0000 7f44e2f1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.738+0000 7f44e2f1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc071fd0 con 0x7f44dc131830 2024-03-27T08:23:11.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e0cb9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e0cb9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34758/0 (socket says 172.21.15.97:34758) 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e0cb9640 1 -- 172.21.15.97:0/1034559920 learned_addr learned my addr 172.21.15.97:0/1034559920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e0cb9640 1 -- 172.21.15.97:0/1034559920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc12f270 con 0x7f44dc131830 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e0cb9640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc070b80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f44d000eda0 con 0x7f44dc131830 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f44d0015420 con 0x7f44dc131830 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc11d650 con 0x7f44dc131830 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f44d00156c0 con 0x7f44dc131830 2024-03-27T08:23:11.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.739+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc11eeb0 con 0x7f44dc131830 2024-03-27T08:23:11.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfc35d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 0x7f0cf411b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfc35d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf411c350 con 0x7f0cf4131880 2024-03-27T08:23:11.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfa0f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 0x7f0cf411b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfa0f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 0x7f0cf411b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34762/0 (socket says 172.21.15.97:34762) 2024-03-27T08:23:11.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfa0f9640 1 -- 172.21.15.97:0/3593789252 learned_addr learned my addr 172.21.15.97:0/3593789252 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:11.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfa0f9640 1 -- 172.21.15.97:0/3593789252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf411d270 con 0x7f0cf4131880 2024-03-27T08:23:11.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0cfa0f9640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 0x7f0cf411b050 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0cf0012af0 con 0x7f0cf4131880 2024-03-27T08:23:11.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0cf0012c50 con 0x7f0cf4131880 2024-03-27T08:23:11.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.740+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0cf0013040 con 0x7f0cf4131880 2024-03-27T08:23:11.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf4074e60 con 0x7f0cf4131880 2024-03-27T08:23:11.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f44d00158c0 con 0x7f44dc131830 2024-03-27T08:23:11.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf411d840 con 0x7f0cf4131880 2024-03-27T08:23:11.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f44d97fa640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f44c00387a0 0x7f44c003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f44d0049660 con 0x7f44dc131830 2024-03-27T08:23:11.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc063450 con 0x7f44dc131830 2024-03-27T08:23:11.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.741+0000 7f44dbfff640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f44c00387a0 0x7f44c003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.742+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f0cf00131f0 con 0x7f0cf4131880 2024-03-27T08:23:11.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.742+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf4063450 con 0x7f0cf4131880 2024-03-27T08:23:11.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.742+0000 7f44dbfff640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f44c00387a0 0x7f44c003acb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.742+0000 7f0ceaffd640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0cd80387a0 0x7f0cd803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.742+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f0cf0018040 con 0x7f0cf4131880 2024-03-27T08:23:11.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.742+0000 7f0cf98f8640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0cd80387a0 0x7f0cd803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.743+0000 7f0cf98f8640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0cd80387a0 0x7f0cd803acb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.747+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f44d0010c00 con 0x7f44dc131830 2024-03-27T08:23:11.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.757+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f0cf0007040 con 0x7f0cf4131880 2024-03-27T08:23:11.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.768+0000 7fae8c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.774+0000 7fae8c6ad640 1 Processor -- start 2024-03-27T08:23:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.774+0000 7fae8c6ad640 1 -- start start 2024-03-27T08:23:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae8c6ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae84131650 0x7fae84131a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae8c6ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae840a06a0 con 0x7fae841345b0 2024-03-27T08:23:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae8c6ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fae840a03e0 con 0x7fae84131650 2024-03-27T08:23:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae8a449640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae84131650 0x7fae84131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae89c48640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fae841345b0 0x7fae841349c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48038/0 (socket says 172.21.15.97:48038) 2024-03-27T08:23:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae89c48640 1 -- 172.21.15.97:0/3065200126 learned_addr learned my addr 172.21.15.97:0/3065200126 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:11.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.775+0000 7fae8a449640 1 -- 172.21.15.97:0/3065200126 >> v1:172.21.15.97:6789/0 conn(0x7fae841345b0 legacy=0x7fae841349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.776+0000 7fae8a449640 1 -- 172.21.15.97:0/3065200126 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8406de30 con 0x7fae84131650 2024-03-27T08:23:11.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.776+0000 7fae8a449640 1 --2- 172.21.15.97:0/3065200126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84131a60 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d891842510e0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.776+0000 7fae89447640 1 -- 172.21.15.97:0/3065200126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fae80009d90 con 0x7fae84131650 2024-03-27T08:23:11.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.776+0000 7fae89447640 1 -- 172.21.15.97:0/3065200126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fae8000e230 con 0x7fae84131650 2024-03-27T08:23:11.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.776+0000 7fae89447640 1 -- 172.21.15.97:0/3065200126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fae8000e4d0 con 0x7fae84131650 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 -- 172.21.15.97:0/3065200126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 msgr2=0x7fae84131a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 --2- 172.21.15.97:0/3065200126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84131a60 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 -- 172.21.15.97:0/3065200126 shutdown_connections 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 --2- 172.21.15.97:0/3065200126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84131a60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 -- 172.21.15.97:0/3065200126 >> 172.21.15.97:0/3065200126 conn(0x7fae840a15e0 msgr2=0x7fae840a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 -- 172.21.15.97:0/3065200126 shutdown_connections 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 1 -- 172.21.15.97:0/3065200126 wait complete. 2024-03-27T08:23:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.777+0000 7fae8c6ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:11.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.779+0000 7fae8c6ad640 1 Processor -- start 2024-03-27T08:23:11.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.779+0000 7fae8c6ad640 1 -- start start 2024-03-27T08:23:11.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8c6ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84120000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8c6ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae84131200 con 0x7fae84131650 2024-03-27T08:23:11.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8a449640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84120000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8a449640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84120000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34772/0 (socket says 172.21.15.97:34772) 2024-03-27T08:23:11.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8a449640 1 -- 172.21.15.97:0/824662505 learned_addr learned my addr 172.21.15.97:0/824662505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:11.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8a449640 1 -- 172.21.15.97:0/824662505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae84120e20 con 0x7fae84131650 2024-03-27T08:23:11.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8a449640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84120000 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fae80014620 con 0x7fae84131650 2024-03-27T08:23:11.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fae80014780 con 0x7fae84131650 2024-03-27T08:23:11.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.780+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841208a0 con 0x7fae84131650 2024-03-27T08:23:11.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.781+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fae80014a20 con 0x7fae84131650 2024-03-27T08:23:11.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.781+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8411f870 con 0x7fae84131650 2024-03-27T08:23:11.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.781+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fae80014bd0 con 0x7fae84131650 2024-03-27T08:23:11.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.782+0000 7fae7affd640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fae680387f0 0x7fae6803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:11.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.782+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fae80019040 con 0x7fae84131650 2024-03-27T08:23:11.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.782+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae84120c20 con 0x7fae84131650 2024-03-27T08:23:11.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.782+0000 7fae89c48640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fae680387f0 0x7fae6803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:11.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.783+0000 7fae89c48640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fae680387f0 0x7fae6803ad00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:11.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.787+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fae80012390 con 0x7fae84131650 2024-03-27T08:23:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.937+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f44dc068a10 con 0x7f44dc131830 2024-03-27T08:23:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.938+0000 7f44d97fa640 1 -- 172.21.15.97:0/1034559920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f44d0010820 con 0x7f44dc131830 2024-03-27T08:23:11.930 INFO:teuthology.orchestra.run.smithi097.stdout:38654705669 2024-03-27T08:23:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.940+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f44c00387a0 msgr2=0x7f44c003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f44c00387a0 0x7f44c003acb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 msgr2=0x7f44dc070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc070b80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 shutdown_connections 2024-03-27T08:23:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f44c00387a0 0x7f44c003acb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 --2- 172.21.15.97:0/1034559920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44dc131830 0x7f44dc070b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 >> 172.21.15.97:0/1034559920 conn(0x7f44dc0a4ae0 msgr2=0x7f44dc1bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 shutdown_connections 2024-03-27T08:23:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.941+0000 7f44e2f1d640 1 -- 172.21.15.97:0/1034559920 wait complete. 2024-03-27T08:23:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.943+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0cf4135650 con 0x7f0cf4131880 2024-03-27T08:23:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.943+0000 7f0ceaffd640 1 -- 172.21.15.97:0/3593789252 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f0cf0016120 con 0x7f0cf4131880 2024-03-27T08:23:11.936 INFO:teuthology.orchestra.run.smithi097.stdout:38654705669 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.945+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0cd80387a0 msgr2=0x7f0cd803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.945+0000 7f0cfc35d640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0cd80387a0 0x7f0cd803acb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.945+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 msgr2=0x7f0cf411b050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 0x7f0cf411b050 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 shutdown_connections 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0cd80387a0 0x7f0cd803acb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 --2- 172.21.15.97:0/3593789252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf4131880 0x7f0cf411b050 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 >> 172.21.15.97:0/3593789252 conn(0x7f0cf40a2900 msgr2=0x7f0cf41bcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 shutdown_connections 2024-03-27T08:23:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.946+0000 7f0cfc35d640 1 -- 172.21.15.97:0/3593789252 wait complete. 2024-03-27T08:23:11.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-03-27T08:23:11.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-03-27T08:23:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.975+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fae8411fac0 con 0x7fae84131650 2024-03-27T08:23:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.975+0000 7fae7affd640 1 -- 172.21.15.97:0/824662505 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7fae8000fb30 con 0x7fae84131650 2024-03-27T08:23:11.968 INFO:teuthology.orchestra.run.smithi097.stdout:38654705669 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fae680387f0 msgr2=0x7fae6803ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fae680387f0 0x7fae6803ad00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 msgr2=0x7fae84120000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84120000 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 shutdown_connections 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fae680387f0 0x7fae6803ad00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 --2- 172.21.15.97:0/824662505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae84131650 0x7fae84120000 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 >> 172.21.15.97:0/824662505 conn(0x7fae840a15e0 msgr2=0x7fae841bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.977+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 shutdown_connections 2024-03-27T08:23:11.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:11.978+0000 7fae8c6ad640 1 -- 172.21.15.97:0/824662505 wait complete. 2024-03-27T08:23:11.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-03-27T08:23:12.945 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-27T08:23:12.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-27T08:23:12.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-27T08:23:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.074+0000 7f3bea82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.083+0000 7f3bea82d640 1 Processor -- start 2024-03-27T08:23:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.083+0000 7f3bea82d640 1 -- start start 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3bea82d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3be4131830 0x7f3be4131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3bea82d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be40735e0 con 0x7f3be4134580 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3bea82d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3be40a0260 con 0x7f3be4131830 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3be3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3be4131830 0x7f3be4131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3be37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3be4134580 0x7f3be4134990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48042/0 (socket says 172.21.15.97:48042) 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3be37fe640 1 -- 172.21.15.97:0/1314274869 learned_addr learned my addr 172.21.15.97:0/1314274869 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3be3fff640 1 -- 172.21.15.97:0/1314274869 >> v1:172.21.15.97:6789/0 conn(0x7f3be4134580 legacy=0x7f3be4134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.085+0000 7f3be3fff640 1 -- 172.21.15.97:0/1314274869 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be406eb80 con 0x7f3be4131830 2024-03-27T08:23:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.086+0000 7f3be3fff640 1 --2- 172.21.15.97:0/1314274869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4131c40 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec17038805a1a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.086+0000 7f3be2ffd640 1 -- 172.21.15.97:0/1314274869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3bd4009d90 con 0x7f3be4131830 2024-03-27T08:23:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.086+0000 7f3be2ffd640 1 -- 172.21.15.97:0/1314274869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3bd400e230 con 0x7f3be4131830 2024-03-27T08:23:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.086+0000 7f3be2ffd640 1 -- 172.21.15.97:0/1314274869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3bd400e4d0 con 0x7f3be4131830 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 -- 172.21.15.97:0/1314274869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 msgr2=0x7f3be4131c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 --2- 172.21.15.97:0/1314274869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4131c40 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 -- 172.21.15.97:0/1314274869 shutdown_connections 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 --2- 172.21.15.97:0/1314274869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4131c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 -- 172.21.15.97:0/1314274869 >> 172.21.15.97:0/1314274869 conn(0x7f3be40a4ae0 msgr2=0x7f3be40a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 -- 172.21.15.97:0/1314274869 shutdown_connections 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.087+0000 7f3bea82d640 1 -- 172.21.15.97:0/1314274869 wait complete. 2024-03-27T08:23:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.088+0000 7f3bea82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.090+0000 7f3bea82d640 1 Processor -- start 2024-03-27T08:23:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.090+0000 7f3bea82d640 1 -- start start 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3bea82d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3bea82d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4071fd0 con 0x7f3be4131830 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34790/0 (socket says 172.21.15.97:34790) 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3be3fff640 1 -- 172.21.15.97:0/3289776477 learned_addr learned my addr 172.21.15.97:0/3289776477 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3be3fff640 1 -- 172.21.15.97:0/3289776477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be412f270 con 0x7f3be4131830 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.092+0000 7f3be3fff640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4070b80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.093+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3bd4014620 con 0x7f3be4131830 2024-03-27T08:23:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.093+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3bd4014780 con 0x7f3be4131830 2024-03-27T08:23:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.093+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be411d650 con 0x7f3be4131830 2024-03-27T08:23:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.093+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3bd4014a20 con 0x7f3be4131830 2024-03-27T08:23:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.093+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be411eeb0 con 0x7f3be4131830 2024-03-27T08:23:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.094+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f3bd4014bd0 con 0x7f3be4131830 2024-03-27T08:23:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.094+0000 7f3be0ff9640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3bd00387a0 0x7f3bd003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.094+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f3bd4019040 con 0x7f3be4131830 2024-03-27T08:23:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.094+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be4063450 con 0x7f3be4131830 2024-03-27T08:23:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.094+0000 7f3be37fe640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3bd00387a0 0x7f3bd003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.095+0000 7f3be37fe640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3bd00387a0 0x7f3bd003acb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.100+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f3bd4011380 con 0x7f3be4131830 2024-03-27T08:23:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.112+0000 7ff38a4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.118+0000 7ff38a4ad640 1 Processor -- start 2024-03-27T08:23:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.118+0000 7ff38a4ad640 1 -- start start 2024-03-27T08:23:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff38a4ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3841378c0 0x7ff384139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff38a4ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3840a06a0 con 0x7ff384131650 2024-03-27T08:23:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff38a4ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff384132170 con 0x7ff3841378c0 2024-03-27T08:23:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff3837fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3841378c0 0x7ff384139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff3837fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3841378c0 0x7ff384139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34798/0 (socket says 172.21.15.97:34798) 2024-03-27T08:23:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff3837fe640 1 -- 172.21.15.97:0/141905991 learned_addr learned my addr 172.21.15.97:0/141905991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.119+0000 7ff383fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff384131650 0x7ff384131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48044/0 (socket says 172.21.15.97:48044) 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.120+0000 7ff3837fe640 1 -- 172.21.15.97:0/141905991 >> v1:172.21.15.97:6789/0 conn(0x7ff384131650 legacy=0x7ff384131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.120+0000 7ff3837fe640 1 -- 172.21.15.97:0/141905991 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38406de30 con 0x7ff3841378c0 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.120+0000 7ff3837fe640 1 --2- 172.21.15.97:0/141905991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841378c0 0x7ff384139ce0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df9f12e10abe9f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.120+0000 7ff382ffd640 1 -- 172.21.15.97:0/141905991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff378009e50 con 0x7ff3841378c0 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.120+0000 7ff382ffd640 1 -- 172.21.15.97:0/141905991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff37800e230 con 0x7ff3841378c0 2024-03-27T08:23:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.120+0000 7ff382ffd640 1 -- 172.21.15.97:0/141905991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff37800e4d0 con 0x7ff3841378c0 2024-03-27T08:23:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 -- 172.21.15.97:0/141905991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841378c0 msgr2=0x7ff384139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 --2- 172.21.15.97:0/141905991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841378c0 0x7ff384139ce0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 -- 172.21.15.97:0/141905991 shutdown_connections 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 --2- 172.21.15.97:0/141905991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3841378c0 0x7ff384139ce0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 -- 172.21.15.97:0/141905991 >> 172.21.15.97:0/141905991 conn(0x7ff3840a15e0 msgr2=0x7ff3840a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 -- 172.21.15.97:0/141905991 shutdown_connections 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 1 -- 172.21.15.97:0/141905991 wait complete. 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7ff38a4ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.122+0000 7ff38a4ad640 1 Processor -- start 2024-03-27T08:23:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.122+0000 7ff38a4ad640 1 -- start start 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.123+0000 7ff38a4ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 0x7ff3841c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.123+0000 7ff38a4ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff384121cc0 con 0x7ff384131650 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff383fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 0x7ff3841c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff383fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 0x7ff3841c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34814/0 (socket says 172.21.15.97:34814) 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff383fff640 1 -- 172.21.15.97:0/586121035 learned_addr learned my addr 172.21.15.97:0/586121035 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff383fff640 1 -- 172.21.15.97:0/586121035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff384120950 con 0x7ff384131650 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff383fff640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 0x7ff3841c6290 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff3740059a0 con 0x7ff384131650 2024-03-27T08:23:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff374005b00 con 0x7ff384131650 2024-03-27T08:23:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3841c5be0 con 0x7ff384131650 2024-03-27T08:23:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff374005da0 con 0x7ff384131650 2024-03-27T08:23:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.124+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3841200f0 con 0x7ff384131650 2024-03-27T08:23:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.125+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff3740070b0 con 0x7ff384131650 2024-03-27T08:23:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.125+0000 7ff380ff9640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3700387a0 0x7ff37003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.126+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff374047fb0 con 0x7ff384131650 2024-03-27T08:23:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.126+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3841c6090 con 0x7ff384131650 2024-03-27T08:23:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.126+0000 7ff3837fe640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3700387a0 0x7ff37003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.126+0000 7ff3837fe640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3700387a0 0x7ff37003acb0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.121+0000 7f7f02d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.128+0000 7f7f02d5d640 1 Processor -- start 2024-03-27T08:23:13.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.128+0000 7f7f02d5d640 1 -- start start 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7f02d5d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7efc137890 0x7f7efc139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7f02d5d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc0735e0 con 0x7f7efc131830 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7f02d5d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7efc0a0260 con 0x7f7efc137890 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7f00af9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7efc131830 0x7f7efc131c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48054/0 (socket says 172.21.15.97:48054) 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7efbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7efc137890 0x7f7efc139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7f00af9640 1 -- 172.21.15.97:0/486026370 learned_addr learned my addr 172.21.15.97:0/486026370 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7efbfff640 1 -- 172.21.15.97:0/486026370 >> v1:172.21.15.97:6789/0 conn(0x7f7efc131830 legacy=0x7f7efc131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.129+0000 7f7efbfff640 1 -- 172.21.15.97:0/486026370 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc06eb80 con 0x7f7efc137890 2024-03-27T08:23:13.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.130+0000 7f7efbfff640 1 --2- 172.21.15.97:0/486026370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc137890 0x7f7efc139cb0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478ba79a25239080 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.132+0000 7f7efb7fe640 1 -- 172.21.15.97:0/486026370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7ee8009d90 con 0x7f7efc137890 2024-03-27T08:23:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.132+0000 7f7efb7fe640 1 -- 172.21.15.97:0/486026370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7ee800e230 con 0x7f7efc137890 2024-03-27T08:23:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.132+0000 7f7efb7fe640 1 -- 172.21.15.97:0/486026370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7ee800e4d0 con 0x7f7efc137890 2024-03-27T08:23:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.132+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff37400d330 con 0x7ff384131650 2024-03-27T08:23:13.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.132+0000 7f7f02d5d640 1 -- 172.21.15.97:0/486026370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc137890 msgr2=0x7f7efc139cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.132+0000 7f7f02d5d640 1 --2- 172.21.15.97:0/486026370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc137890 0x7f7efc139cb0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.133+0000 7f7f02d5d640 1 -- 172.21.15.97:0/486026370 shutdown_connections 2024-03-27T08:23:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.133+0000 7f7f02d5d640 1 --2- 172.21.15.97:0/486026370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc137890 0x7f7efc139cb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.133+0000 7f7f02d5d640 1 -- 172.21.15.97:0/486026370 >> 172.21.15.97:0/486026370 conn(0x7f7efc0a4ae0 msgr2=0x7f7efc0a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.133+0000 7f7f02d5d640 1 -- 172.21.15.97:0/486026370 shutdown_connections 2024-03-27T08:23:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.133+0000 7f7f02d5d640 1 -- 172.21.15.97:0/486026370 wait complete. 2024-03-27T08:23:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.133+0000 7f7f02d5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.134+0000 7f7f02d5d640 1 Processor -- start 2024-03-27T08:23:13.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.134+0000 7f7f02d5d640 1 -- start start 2024-03-27T08:23:13.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.135+0000 7f7f02d5d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 0x7f7efc1c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.135+0000 7f7f02d5d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc071ec0 con 0x7f7efc131830 2024-03-27T08:23:13.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f00af9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 0x7f7efc1c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f00af9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 0x7f7efc1c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34832/0 (socket says 172.21.15.97:34832) 2024-03-27T08:23:13.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f00af9640 1 -- 172.21.15.97:0/736756994 learned_addr learned my addr 172.21.15.97:0/736756994 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:13.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f00af9640 1 -- 172.21.15.97:0/736756994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1c91c0 con 0x7f7efc131830 2024-03-27T08:23:13.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f00af9640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 0x7f7efc1c9790 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7ef0005a60 con 0x7f7efc131830 2024-03-27T08:23:13.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7ef0005bc0 con 0x7f7efc131830 2024-03-27T08:23:13.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1c93f0 con 0x7f7efc131830 2024-03-27T08:23:13.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7ef0005e60 con 0x7f7efc131830 2024-03-27T08:23:13.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.136+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc11d9f0 con 0x7f7efc131830 2024-03-27T08:23:13.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.137+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7ef00070b0 con 0x7f7efc131830 2024-03-27T08:23:13.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.138+0000 7f7ef97fa640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7ee0038750 0x7f7ee003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.138+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f7ef0047f80 con 0x7f7efc131830 2024-03-27T08:23:13.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.138+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc11d2c0 con 0x7f7efc131830 2024-03-27T08:23:13.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.138+0000 7f7efbfff640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7ee0038750 0x7f7ee003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.139+0000 7f7efbfff640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7ee0038750 0x7f7ee003ac60 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.143+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f7ef000d710 con 0x7f7efc131830 2024-03-27T08:23:13.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.306+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3be4068a10 con 0x7f3be4131830 2024-03-27T08:23:13.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.307+0000 7f3be0ff9640 1 -- 172.21.15.97:0/3289776477 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f3bd4010310 con 0x7f3be4131830 2024-03-27T08:23:13.299 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3bd00387a0 msgr2=0x7f3bd003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3bd00387a0 0x7f3bd003acb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 msgr2=0x7f3be4070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4070b80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 shutdown_connections 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3bd00387a0 0x7f3bd003acb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 --2- 172.21.15.97:0/3289776477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be4131830 0x7f3be4070b80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 >> 172.21.15.97:0/3289776477 conn(0x7f3be40a4ae0 msgr2=0x7f3be41bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 shutdown_connections 2024-03-27T08:23:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.309+0000 7f3bea82d640 1 -- 172.21.15.97:0/3289776477 wait complete. 2024-03-27T08:23:13.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.320+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff384063450 con 0x7ff384131650 2024-03-27T08:23:13.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2024-03-27T08:23:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.320+0000 7ff380ff9640 1 -- 172.21.15.97:0/586121035 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7ff37400d150 con 0x7ff384131650 2024-03-27T08:23:13.313 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:13.314 DEBUG:teuthology.parallel:result is None 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3700387a0 msgr2=0x7ff37003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3700387a0 0x7ff37003acb0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 msgr2=0x7ff3841c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 0x7ff3841c6290 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 shutdown_connections 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff3700387a0 0x7ff37003acb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 --2- 172.21.15.97:0/586121035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff384131650 0x7ff3841c6290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 >> 172.21.15.97:0/586121035 conn(0x7ff3840a15e0 msgr2=0x7ff3840a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 shutdown_connections 2024-03-27T08:23:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.322+0000 7ff38a4ad640 1 -- 172.21.15.97:0/586121035 wait complete. 2024-03-27T08:23:13.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-03-27T08:23:13.325 DEBUG:teuthology.parallel:result is None 2024-03-27T08:23:13.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.340+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7efc063450 con 0x7f7efc131830 2024-03-27T08:23:13.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.341+0000 7f7ef97fa640 1 -- 172.21.15.97:0/736756994 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f7ef000d150 con 0x7f7efc131830 2024-03-27T08:23:13.333 INFO:teuthology.orchestra.run.smithi097.stdout:38654705670 2024-03-27T08:23:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7ee0038750 msgr2=0x7f7ee003ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7ee0038750 0x7f7ee003ac60 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 msgr2=0x7f7efc1c9790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 0x7f7efc1c9790 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 shutdown_connections 2024-03-27T08:23:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7ee0038750 0x7f7ee003ac60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 --2- 172.21.15.97:0/736756994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc131830 0x7f7efc1c9790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 >> 172.21.15.97:0/736756994 conn(0x7f7efc0a4ae0 msgr2=0x7f7efc1bc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 shutdown_connections 2024-03-27T08:23:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.343+0000 7f7f02d5d640 1 -- 172.21.15.97:0/736756994 wait complete. 2024-03-27T08:23:13.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2024-03-27T08:23:13.346 DEBUG:teuthology.parallel:result is None 2024-03-27T08:23:13.346 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-27T08:23:13.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:23:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.499+0000 7ff5626ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.504+0000 7ff5626ad640 1 Processor -- start 2024-03-27T08:23:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.504+0000 7ff5626ad640 1 -- start start 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff5626ad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff55c1378c0 0x7ff55c139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff5626ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff55c0a06a0 con 0x7ff55c131650 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff5626ad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff55c132170 con 0x7ff55c1378c0 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff55c1378c0 0x7ff55c139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff55c131650 0x7ff55c131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48066/0 (socket says 172.21.15.97:48066) 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff55c1378c0 0x7ff55c139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34844/0 (socket says 172.21.15.97:34844) 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55bfff640 1 -- 172.21.15.97:0/301535005 learned_addr learned my addr 172.21.15.97:0/301535005 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55b7fe640 1 -- 172.21.15.97:0/301535005 >> v1:172.21.15.97:6789/0 conn(0x7ff55c131650 legacy=0x7ff55c131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55b7fe640 1 -- 172.21.15.97:0/301535005 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c06de30 con 0x7ff55c1378c0 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55b7fe640 1 --2- 172.21.15.97:0/301535005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c1378c0 0x7ff55c139ce0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0a1f2332dea7cd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55affd640 1 -- 172.21.15.97:0/301535005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff550009d90 con 0x7ff55c1378c0 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55affd640 1 -- 172.21.15.97:0/301535005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff55000e230 con 0x7ff55c1378c0 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.505+0000 7ff55affd640 1 -- 172.21.15.97:0/301535005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff55000e4d0 con 0x7ff55c1378c0 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 -- 172.21.15.97:0/301535005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c1378c0 msgr2=0x7ff55c139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 --2- 172.21.15.97:0/301535005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c1378c0 0x7ff55c139ce0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 -- 172.21.15.97:0/301535005 shutdown_connections 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 --2- 172.21.15.97:0/301535005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c1378c0 0x7ff55c139ce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 -- 172.21.15.97:0/301535005 >> 172.21.15.97:0/301535005 conn(0x7ff55c0a15e0 msgr2=0x7ff55c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 -- 172.21.15.97:0/301535005 shutdown_connections 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.506+0000 7ff5626ad640 1 -- 172.21.15.97:0/301535005 wait complete. 2024-03-27T08:23:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.507+0000 7ff5626ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.508+0000 7ff5626ad640 1 Processor -- start 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.508+0000 7ff5626ad640 1 -- start start 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.509+0000 7ff5626ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 0x7ff55c1c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.509+0000 7ff5626ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c121cc0 con 0x7ff55c131650 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.509+0000 7ff55bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 0x7ff55c1c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.509+0000 7ff55bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 0x7ff55c1c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34856/0 (socket says 172.21.15.97:34856) 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.509+0000 7ff55bfff640 1 -- 172.21.15.97:0/2254971083 learned_addr learned my addr 172.21.15.97:0/2254971083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.509+0000 7ff55bfff640 1 -- 172.21.15.97:0/2254971083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c120950 con 0x7ff55c131650 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.510+0000 7ff55bfff640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 0x7ff55c1c6290 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.510+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff54c005a60 con 0x7ff55c131650 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.510+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff54c005bc0 con 0x7ff55c131650 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.510+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c1c5be0 con 0x7ff55c131650 2024-03-27T08:23:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.510+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff54c005e60 con 0x7ff55c131650 2024-03-27T08:23:13.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.510+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c1200f0 con 0x7ff55c131650 2024-03-27T08:23:13.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.511+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff54c00c780 con 0x7ff55c131650 2024-03-27T08:23:13.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.511+0000 7ff558ff9640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff5480387a0 0x7ff54803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.511+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff54c047fb0 con 0x7ff55c131650 2024-03-27T08:23:13.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.511+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c1c6090 con 0x7ff55c131650 2024-03-27T08:23:13.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.511+0000 7ff55b7fe640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff5480387a0 0x7ff54803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.512+0000 7ff55b7fe640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff5480387a0 0x7ff54803acb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.516+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff54c00d330 con 0x7ff55c131650 2024-03-27T08:23:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.689+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff55c0a0f20 con 0x7ff5480387a0 2024-03-27T08:23:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.691+0000 7ff558ff9640 1 -- 172.21.15.97:0/2254971083 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+29954 (crc 0 0 0) 0x7ff55000e3a0 con 0x7ff5480387a0 2024-03-27T08:23:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff5480387a0 msgr2=0x7ff54803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff5480387a0 0x7ff54803acb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 msgr2=0x7ff55c1c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 0x7ff55c1c6290 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 shutdown_connections 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff5480387a0 0x7ff54803acb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 --2- 172.21.15.97:0/2254971083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55c131650 0x7ff55c1c6290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 >> 172.21.15.97:0/2254971083 conn(0x7ff55c0a15e0 msgr2=0x7ff55c0a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 shutdown_connections 2024-03-27T08:23:13.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.693+0000 7ff5626ad640 1 -- 172.21.15.97:0/2254971083 wait complete. 2024-03-27T08:23:13.696 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-03-27T08:23:11.927240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.749953"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.065652+0000","last_change":"2024-03-27T08:23:09.066891+0000","last_active":"2024-03-27T08:23:09.065652+0000","last_peered":"2024-03-27T08:23:09.065652+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:23:09.065652+0000","last_undegraded":"2024-03-27T08:23:09.065652+0000","last_fullsized":"2024-03-27T08:23:09.065652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.058320+0000","last_change":"2024-03-27T08:23:09.059885+0000","last_active":"2024-03-27T08:23:09.058320+0000","last_peered":"2024-03-27T08:23:09.058320+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:23:09.058320+0000","last_undegraded":"2024-03-27T08:23:09.058320+0000","last_fullsized":"2024-03-27T08:23:09.058320+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:23:10.018664+0000","last_change":"2024-03-27T08:23:09.052784+0000","last_active":"2024-03-27T08:23:10.018664+0000","last_peered":"2024-03-27T08:23:10.018664+0000","last_clean":"2024-03-27T08:23:10.018664+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:23:10.018664+0000","last_undegraded":"2024-03-27T08:23:10.018664+0000","last_fullsized":"2024-03-27T08:23:10.018664+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.052344+0000","last_change":"2024-03-27T08:23:09.053884+0000","last_active":"2024-03-27T08:23:09.052344+0000","last_peered":"2024-03-27T08:23:09.052344+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:23:09.052344+0000","last_undegraded":"2024-03-27T08:23:09.052344+0000","last_fullsized":"2024-03-27T08:23:09.052344+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.060453+0000","last_change":"2024-03-27T08:23:09.061623+0000","last_active":"2024-03-27T08:23:09.060453+0000","last_peered":"2024-03-27T08:23:09.060453+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:23:09.060453+0000","last_undegraded":"2024-03-27T08:23:09.060453+0000","last_fullsized":"2024-03-27T08:23:09.060453+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.053616+0000","last_change":"2024-03-27T08:23:09.055461+0000","last_active":"2024-03-27T08:23:09.053616+0000","last_peered":"2024-03-27T08:23:09.053616+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:23:09.053616+0000","last_undegraded":"2024-03-27T08:23:09.053616+0000","last_fullsized":"2024-03-27T08:23:09.053616+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.073654+0000","last_change":"2024-03-27T08:23:09.074614+0000","last_active":"2024-03-27T08:23:09.073654+0000","last_peered":"2024-03-27T08:23:09.073654+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:23:09.073654+0000","last_undegraded":"2024-03-27T08:23:09.073654+0000","last_fullsized":"2024-03-27T08:23:09.073654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.064129+0000","last_change":"2024-03-27T08:23:09.065607+0000","last_active":"2024-03-27T08:23:09.064129+0000","last_peered":"2024-03-27T08:23:09.064129+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:23:09.064129+0000","last_undegraded":"2024-03-27T08:23:09.064129+0000","last_fullsized":"2024-03-27T08:23:09.064129+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:23:10.015413+0000","last_change":"2024-03-27T08:23:07.031764+0000","last_active":"2024-03-27T08:23:10.015413+0000","last_peered":"2024-03-27T08:23:10.015413+0000","last_clean":"2024-03-27T08:23:10.015413+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:23:10.015413+0000","last_undegraded":"2024-03-27T08:23:10.015413+0000","last_fullsized":"2024-03-27T08:23:10.015413+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:23:13.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:23:13.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.810+0000 7ff22691d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.814+0000 7ff22691d640 1 Processor -- start 2024-03-27T08:23:13.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.814+0000 7ff22691d640 1 -- start start 2024-03-27T08:23:13.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.815+0000 7ff22691d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff220134630 0x7ff220136a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.815+0000 7ff22691d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22006e090 con 0x7ff22012e2c0 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.815+0000 7ff22691d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff22009ef70 con 0x7ff220134630 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff220134630 0x7ff220136a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff22012e2c0 0x7ff22012e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48082/0 (socket says 172.21.15.97:48082) 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21ffff640 1 -- 172.21.15.97:0/2154720816 learned_addr learned my addr 172.21.15.97:0/2154720816 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff220134630 0x7ff220136a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34868/0 (socket says 172.21.15.97:34868) 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21f7fe640 1 -- 172.21.15.97:0/2154720816 >> v1:172.21.15.97:6789/0 conn(0x7ff22012e2c0 legacy=0x7ff22012e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21effd640 1 -- 172.21.15.97:0/2154720816 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614985152 0 0) 0x7ff22006e090 con 0x7ff22012e2c0 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21f7fe640 1 -- 172.21.15.97:0/2154720816 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2200734e0 con 0x7ff220134630 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21f7fe640 1 --2- 172.21.15.97:0/2154720816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220134630 0x7ff220136a50 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9005dd3fa1ebd221 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21effd640 1 -- 172.21.15.97:0/2154720816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff214009d90 con 0x7ff220134630 2024-03-27T08:23:13.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21effd640 1 -- 172.21.15.97:0/2154720816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff21400e230 con 0x7ff220134630 2024-03-27T08:23:13.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.816+0000 7ff21effd640 1 -- 172.21.15.97:0/2154720816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff21400e4d0 con 0x7ff220134630 2024-03-27T08:23:13.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 -- 172.21.15.97:0/2154720816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220134630 msgr2=0x7ff220136a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 --2- 172.21.15.97:0/2154720816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220134630 0x7ff220136a50 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 -- 172.21.15.97:0/2154720816 shutdown_connections 2024-03-27T08:23:13.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 --2- 172.21.15.97:0/2154720816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220134630 0x7ff220136a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 -- 172.21.15.97:0/2154720816 >> 172.21.15.97:0/2154720816 conn(0x7ff22009d1d0 msgr2=0x7ff22009d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 -- 172.21.15.97:0/2154720816 shutdown_connections 2024-03-27T08:23:13.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 1 -- 172.21.15.97:0/2154720816 wait complete. 2024-03-27T08:23:13.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.817+0000 7ff22691d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:13.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.818+0000 7ff22691d640 1 Processor -- start 2024-03-27T08:23:13.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.819+0000 7ff22691d640 1 -- start start 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff22691d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 0x7ff22011a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff22691d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff22011ea80 con 0x7ff22012e2c0 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 0x7ff22011a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 0x7ff22011a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34870/0 (socket says 172.21.15.97:34870) 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21ffff640 1 -- 172.21.15.97:0/1938778870 learned_addr learned my addr 172.21.15.97:0/1938778870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21ffff640 1 -- 172.21.15.97:0/1938778870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22006aa50 con 0x7ff22012e2c0 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21ffff640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 0x7ff22011a990 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff21000ebd0 con 0x7ff22012e2c0 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff21000ed30 con 0x7ff22012e2c0 2024-03-27T08:23:13.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22011df70 con 0x7ff22012e2c0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff210004840 con 0x7ff22012e2c0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.820+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22011c520 con 0x7ff22012e2c0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.821+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff2100049f0 con 0x7ff22012e2c0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.821+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22011ce00 con 0x7ff22012e2c0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.821+0000 7ff21cff9640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff20c0387a0 0x7ff20c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.821+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7ff21004a860 con 0x7ff22012e2c0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.822+0000 7ff21f7fe640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff20c0387a0 0x7ff20c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:13.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.822+0000 7ff21f7fe640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff20c0387a0 0x7ff20c03acb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:13.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.827+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff210001a00 con 0x7ff22012e2c0 2024-03-27T08:23:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.997+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff22009ac90 con 0x7ff20c0387a0 2024-03-27T08:23:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:13.999+0000 7ff21cff9640 1 -- 172.21.15.97:0/1938778870 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+29954 (crc 0 0 0) 0x7ff21400e3a0 con 0x7ff20c0387a0 2024-03-27T08:23:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff20c0387a0 msgr2=0x7ff20c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff20c0387a0 0x7ff20c03acb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 msgr2=0x7ff22011a990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 0x7ff22011a990 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 shutdown_connections 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff20c0387a0 0x7ff20c03acb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 --2- 172.21.15.97:0/1938778870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22012e2c0 0x7ff22011a990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 >> 172.21.15.97:0/1938778870 conn(0x7ff22009d1d0 msgr2=0x7ff220132620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 shutdown_connections 2024-03-27T08:23:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.001+0000 7ff22691d640 1 -- 172.21.15.97:0/1938778870 wait complete. 2024-03-27T08:23:14.004 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-03-27T08:23:13.927744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.750458"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.065652+0000","last_change":"2024-03-27T08:23:09.066891+0000","last_active":"2024-03-27T08:23:09.065652+0000","last_peered":"2024-03-27T08:23:09.065652+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:23:09.065652+0000","last_undegraded":"2024-03-27T08:23:09.065652+0000","last_fullsized":"2024-03-27T08:23:09.065652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.058320+0000","last_change":"2024-03-27T08:23:09.059885+0000","last_active":"2024-03-27T08:23:09.058320+0000","last_peered":"2024-03-27T08:23:09.058320+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:23:09.058320+0000","last_undegraded":"2024-03-27T08:23:09.058320+0000","last_fullsized":"2024-03-27T08:23:09.058320+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:23:10.018664+0000","last_change":"2024-03-27T08:23:09.052784+0000","last_active":"2024-03-27T08:23:10.018664+0000","last_peered":"2024-03-27T08:23:10.018664+0000","last_clean":"2024-03-27T08:23:10.018664+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:23:10.018664+0000","last_undegraded":"2024-03-27T08:23:10.018664+0000","last_fullsized":"2024-03-27T08:23:10.018664+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.052344+0000","last_change":"2024-03-27T08:23:09.053884+0000","last_active":"2024-03-27T08:23:09.052344+0000","last_peered":"2024-03-27T08:23:09.052344+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:23:09.052344+0000","last_undegraded":"2024-03-27T08:23:09.052344+0000","last_fullsized":"2024-03-27T08:23:09.052344+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.060453+0000","last_change":"2024-03-27T08:23:09.061623+0000","last_active":"2024-03-27T08:23:09.060453+0000","last_peered":"2024-03-27T08:23:09.060453+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:23:09.060453+0000","last_undegraded":"2024-03-27T08:23:09.060453+0000","last_fullsized":"2024-03-27T08:23:09.060453+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.053616+0000","last_change":"2024-03-27T08:23:09.055461+0000","last_active":"2024-03-27T08:23:09.053616+0000","last_peered":"2024-03-27T08:23:09.053616+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:23:09.053616+0000","last_undegraded":"2024-03-27T08:23:09.053616+0000","last_fullsized":"2024-03-27T08:23:09.053616+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.073654+0000","last_change":"2024-03-27T08:23:09.074614+0000","last_active":"2024-03-27T08:23:09.073654+0000","last_peered":"2024-03-27T08:23:09.073654+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:23:09.073654+0000","last_undegraded":"2024-03-27T08:23:09.073654+0000","last_fullsized":"2024-03-27T08:23:09.073654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-27T08:23:09.064129+0000","last_change":"2024-03-27T08:23:09.065607+0000","last_active":"2024-03-27T08:23:09.064129+0000","last_peered":"2024-03-27T08:23:09.064129+0000","last_clean":"2024-03-27T08:23:07.989354+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:23:09.064129+0000","last_undegraded":"2024-03-27T08:23:09.064129+0000","last_fullsized":"2024-03-27T08:23:09.064129+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-27T08:23:10.015413+0000","last_change":"2024-03-27T08:23:07.031764+0000","last_active":"2024-03-27T08:23:10.015413+0000","last_peered":"2024-03-27T08:23:10.015413+0000","last_clean":"2024-03-27T08:23:10.015413+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:23:10.015413+0000","last_undegraded":"2024-03-27T08:23:10.015413+0000","last_fullsized":"2024-03-27T08:23:10.015413+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:23:14.005 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-27T08:23:14.005 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-27T08:23:14.005 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-27T08:23:14.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-27T08:23:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.118+0000 7fe16da6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.122+0000 7fe16da6d640 1 Processor -- start 2024-03-27T08:23:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.122+0000 7fe16da6d640 1 -- start start 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.123+0000 7fe16da6d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe16806e040 0x7fe168132a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.123+0000 7fe16da6d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16806fbb0 con 0x7fe168132f60 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.123+0000 7fe16da6d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe16806dbc0 con 0x7fe16806e040 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.123+0000 7fe1677fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe16806e040 0x7fe168132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.123+0000 7fe166ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe168132f60 0x7fe16809cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48092/0 (socket says 172.21.15.97:48092) 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.123+0000 7fe166ffd640 1 -- 172.21.15.97:0/4281040467 learned_addr learned my addr 172.21.15.97:0/4281040467 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:23:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.124+0000 7fe1677fe640 1 -- 172.21.15.97:0/4281040467 >> v1:172.21.15.97:6789/0 conn(0x7fe168132f60 legacy=0x7fe16809cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.124+0000 7fe1677fe640 1 -- 172.21.15.97:0/4281040467 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe168132710 con 0x7fe16806e040 2024-03-27T08:23:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.124+0000 7fe1677fe640 1 --2- 172.21.15.97:0/4281040467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe168132a20 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205aac934085bffa server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.124+0000 7fe1667fc640 1 -- 172.21.15.97:0/4281040467 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe158009d90 con 0x7fe16806e040 2024-03-27T08:23:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.124+0000 7fe1667fc640 1 -- 172.21.15.97:0/4281040467 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe15800e230 con 0x7fe16806e040 2024-03-27T08:23:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.124+0000 7fe1667fc640 1 -- 172.21.15.97:0/4281040467 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe15800e4d0 con 0x7fe16806e040 2024-03-27T08:23:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 -- 172.21.15.97:0/4281040467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 msgr2=0x7fe168132a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 --2- 172.21.15.97:0/4281040467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe168132a20 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 -- 172.21.15.97:0/4281040467 shutdown_connections 2024-03-27T08:23:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 --2- 172.21.15.97:0/4281040467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe168132a20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 -- 172.21.15.97:0/4281040467 >> 172.21.15.97:0/4281040467 conn(0x7fe16809e2f0 msgr2=0x7fe16809e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 -- 172.21.15.97:0/4281040467 shutdown_connections 2024-03-27T08:23:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 1 -- 172.21.15.97:0/4281040467 wait complete. 2024-03-27T08:23:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.125+0000 7fe16da6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:23:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.126+0000 7fe16da6d640 1 Processor -- start 2024-03-27T08:23:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.126+0000 7fe16da6d640 1 -- start start 2024-03-27T08:23:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.127+0000 7fe16da6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe1681c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.127+0000 7fe16da6d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe16811e7c0 con 0x7fe16806e040 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.127+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe1681c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.127+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe1681c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34882/0 (socket says 172.21.15.97:34882) 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe1677fe640 1 -- 172.21.15.97:0/184365699 learned_addr learned my addr 172.21.15.97:0/184365699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe1677fe640 1 -- 172.21.15.97:0/184365699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16811fae0 con 0x7fe16806e040 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe1677fe640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe1681c75f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe158014620 con 0x7fe16806e040 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe158014780 con 0x7fe16806e040 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681c7250 con 0x7fe16806e040 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe158014a20 con 0x7fe16806e040 2024-03-27T08:23:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.128+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681c6e60 con 0x7fe16806e040 2024-03-27T08:23:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.129+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe158014bd0 con 0x7fe16806e040 2024-03-27T08:23:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.129+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16811e530 con 0x7fe16806e040 2024-03-27T08:23:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.129+0000 7fe14ffff640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe1540387a0 0x7fe15403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:23:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.129+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fe158019040 con 0x7fe16806e040 2024-03-27T08:23:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.130+0000 7fe166ffd640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe1540387a0 0x7fe15403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:23:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.130+0000 7fe166ffd640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe1540387a0 0x7fe15403acb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:23:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.135+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fe158018190 con 0x7fe16806e040 2024-03-27T08:23:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.347+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe168063450 con 0x7fe16806e040 2024-03-27T08:23:14.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.348+0000 7fe14ffff640 1 -- 172.21.15.97:0/184365699 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (crc 0 0 0) 0x7fe15800fb30 con 0x7fe16806e040 2024-03-27T08:23:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:23:14.340 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-27T08:23:14.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe1540387a0 msgr2=0x7fe15403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:14.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe1540387a0 0x7fe15403acb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:14.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 msgr2=0x7fe1681c75f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe1681c75f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 shutdown_connections 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe1540387a0 0x7fe15403acb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 --2- 172.21.15.97:0/184365699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16806e040 0x7fe1681c75f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 >> 172.21.15.97:0/184365699 conn(0x7fe16809e2f0 msgr2=0x7fe168130ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 shutdown_connections 2024-03-27T08:23:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:23:14.350+0000 7fe16da6d640 1 -- 172.21.15.97:0/184365699 wait complete. 2024-03-27T08:23:14.352 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-27T08:23:14.353 INFO:teuthology.run_tasks:Running task workunit... 2024-03-27T08:23:14.367 INFO:tasks.workunit:Pulling workunits from ref eae914f097640d21427554e7f6aa6f1e9c4a7748 2024-03-27T08:23:14.368 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-27T08:23:14.368 DEBUG:teuthology.orchestra.run.smithi097:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-27T08:23:14.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:23:14.383 INFO:teuthology.orchestra.run.smithi097.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-27T08:23:14.383 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-27T08:23:14.437 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-27T08:23:14.438 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-27T08:23:14.492 INFO:tasks.workunit:timeout=3h 2024-03-27T08:23:14.493 INFO:tasks.workunit:cleanup=True 2024-03-27T08:23:14.493 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout eae914f097640d21427554e7f6aa6f1e9c4a7748 2024-03-27T08:23:14.548 INFO:tasks.workunit.client.0.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-27T08:23:57.344 INFO:tasks.workunit.client.0.smithi097.stderr:Note: switching to 'eae914f097640d21427554e7f6aa6f1e9c4a7748'. 2024-03-27T08:23:57.344 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.344 INFO:tasks.workunit.client.0.smithi097.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-27T08:23:57.344 INFO:tasks.workunit.client.0.smithi097.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr:state without impacting any branches by switching back to a branch. 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: git switch -c 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr:Or undo this operation with: 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: git switch - 2024-03-27T08:23:57.345 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.346 INFO:tasks.workunit.client.0.smithi097.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-27T08:23:57.346 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T08:23:57.346 INFO:tasks.workunit.client.0.smithi097.stderr:HEAD is now at eae914f0976 Merge pull request #56449 from afreen23/wip-security 2024-03-27T08:23:57.346 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-27T08:23:57.407 INFO:tasks.workunit.client.0.smithi097.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-27T08:23:57.409 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T08:23:57.409 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-27T08:23:58.169 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-27T08:23:58.234 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-27T08:23:58.274 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T08:23:58.276 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T08:23:58.276 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-27T08:23:58.315 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T08:23:58.318 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T08:23:58.318 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-27T08:23:58.374 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-27T08:23:58.375 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-27T08:23:58.376 DEBUG:teuthology.orchestra.run.smithi097:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=eae914f097640d21427554e7f6aa6f1e9c4a7748 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ parallel=1 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' --crimson = --serial ']' 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ crimson=0 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' --crimson = --crimson ']' 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ parallel=0 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ crimson=1 2024-03-27T08:23:58.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ color= 2024-03-27T08:23:58.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -t 1 ']' 2024-03-27T08:23:58.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-27T08:23:58.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-27T08:23:58.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-27T08:23:58.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ declare -A pids 2024-03-27T08:23:58.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:23:58.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:23:58.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_aio 2024-03-27T08:23:58.450 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:23:58.450 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 42 tests from 2 test suites. 2024-03-27T08:23:58.450 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:23:58.450 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 26 tests from LibRadosAio 2024-03-27T08:23:58.450 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.TooBig 2024-03-27T08:24:00.654 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.TooBig (2204 ms) 2024-03-27T08:24:00.655 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-03-27T08:24:03.666 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.SimpleWrite (3012 ms) 2024-03-27T08:24:03.666 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-03-27T08:24:06.677 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.WaitForSafe (3011 ms) 2024-03-27T08:24:06.677 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-03-27T08:24:09.688 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTrip (3011 ms) 2024-03-27T08:24:09.688 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-03-27T08:24:12.706 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTrip2 (3017 ms) 2024-03-27T08:24:12.706 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-03-27T08:24:15.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTrip3 (3018 ms) 2024-03-27T08:24:15.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-03-27T08:24:18.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripAppend (3012 ms) 2024-03-27T08:24:18.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-03-27T08:24:21.748 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RemoveTest (3013 ms) 2024-03-27T08:24:21.749 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-27T08:24:24.762 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3014 ms) 2024-03-27T08:24:24.762 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RmXattr 2024-03-27T08:24:27.775 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RmXattr (3013 ms) 2024-03-27T08:24:27.776 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.XattrIter 2024-03-27T08:24:30.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.XattrIter (3019 ms) 2024-03-27T08:24:30.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.IsComplete 2024-03-27T08:24:33.808 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.IsComplete (3014 ms) 2024-03-27T08:24:33.809 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.IsSafe 2024-03-27T08:24:36.826 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.IsSafe (3017 ms) 2024-03-27T08:24:36.826 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-03-27T08:24:39.838 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.ReturnValue (3013 ms) 2024-03-27T08:24:39.839 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.Flush 2024-03-27T08:24:42.858 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.Flush (3020 ms) 2024-03-27T08:24:42.858 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-03-27T08:24:45.869 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.FlushAsync (3011 ms) 2024-03-27T08:24:45.870 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-27T08:24:48.880 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3011 ms) 2024-03-27T08:24:48.881 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-27T08:24:51.895 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (3015 ms) 2024-03-27T08:24:51.895 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-03-27T08:24:54.645 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.SimpleStat (2750 ms) 2024-03-27T08:24:54.645 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-03-27T08:24:57.645 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.OperateMtime (3000 ms) 2024-03-27T08:24:57.645 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-03-27T08:25:00.674 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.Operate2Mtime (3029 ms) 2024-03-27T08:25:00.675 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-03-27T08:25:03.678 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.SimpleStatNS (3004 ms) 2024-03-27T08:25:03.679 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.StatRemove 2024-03-27T08:25:06.692 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.StatRemove (3014 ms) 2024-03-27T08:25:06.692 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-03-27T08:25:09.695 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.ExecuteClass (3003 ms) 2024-03-27T08:25:09.696 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-03-27T08:25:12.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.MultiWrite (3018 ms) 2024-03-27T08:25:12.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.AioUnlock (3015 ms) 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 26 tests from LibRadosAio (77278 ms total) 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 16 tests from LibRadosAioEC 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1100: Skipped 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-03-27T08:25:15.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1132: Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1149: Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1182: Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1215: Skipped 2024-03-27T08:25:15.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1286: Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1326: Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1366: Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-03-27T08:25:15.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.Flush 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1385: Skipped 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1415: Skipped 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1454: Skipped 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-03-27T08:25:15.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1498: Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1532: Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1593: Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1649: Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio.cc:1681: Skipped 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 42 tests from 2 test suites ran. (77278 ms total) 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 26 tests. 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 16 tests, listed below: 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-03-27T08:25:15.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-03-27T08:25:15.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-03-27T08:25:15.738 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:25:15.738 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:25:15.738 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_aio_pp 2024-03-27T08:25:15.750 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:25:15.750 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 57 tests from 4 test suites. 2024-03-27T08:25:15.750 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:25:15.750 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 32 tests from LibRadosAio 2024-03-27T08:25:15.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-03-27T08:25:18.649 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.TooBigPP (2899 ms) 2024-03-27T08:25:18.649 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-03-27T08:25:35.658 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.PoolQuotaPP (17009 ms) 2024-03-27T08:25:35.658 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-03-27T08:25:41.684 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.SimpleWritePP (6025 ms) 2024-03-27T08:25:41.684 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-03-27T08:25:44.694 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.WaitForSafePP (3011 ms) 2024-03-27T08:25:44.694 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-03-27T08:25:47.655 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripPP (2960 ms) 2024-03-27T08:25:47.655 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-03-27T08:25:50.670 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3016 ms) 2024-03-27T08:25:50.671 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-03-27T08:25:53.682 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3012 ms) 2024-03-27T08:25:53.683 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-27T08:25:56.696 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3014 ms) 2024-03-27T08:25:56.696 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-03-27T08:25:59.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.IsCompletePP (3004 ms) 2024-03-27T08:25:59.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-03-27T08:26:02.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.IsSafePP (3019 ms) 2024-03-27T08:26:02.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-03-27T08:26:05.727 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.ReturnValuePP (3008 ms) 2024-03-27T08:26:05.728 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.FlushPP 2024-03-27T08:26:08.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.FlushPP (3015 ms) 2024-03-27T08:26:08.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-03-27T08:26:11.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3009 ms) 2024-03-27T08:26:11.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-27T08:26:14.764 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3013 ms) 2024-03-27T08:26:14.764 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-27T08:26:17.662 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (2898 ms) 2024-03-27T08:26:17.662 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-27T08:26:20.679 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3016 ms) 2024-03-27T08:26:20.679 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-27T08:26:23.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3012 ms) 2024-03-27T08:26:23.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-27T08:26:26.712 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3022 ms) 2024-03-27T08:26:26.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-03-27T08:26:29.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.SimpleStatPP (3001 ms) 2024-03-27T08:26:29.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-03-27T08:26:32.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.OperateMtime (3018 ms) 2024-03-27T08:26:32.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-03-27T08:26:35.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.OperateMtime2 (3011 ms) 2024-03-27T08:26:35.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-03-27T08:26:38.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.StatRemovePP (3016 ms) 2024-03-27T08:26:38.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-03-27T08:26:41.768 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3010 ms) 2024-03-27T08:26:41.768 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.OmapPP 2024-03-27T08:26:44.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.OmapPP (3011 ms) 2024-03-27T08:26:44.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-03-27T08:26:47.666 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.MultiWritePP (2886 ms) 2024-03-27T08:26:47.666 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-03-27T08:26:50.677 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.AioUnlockPP (3012 ms) 2024-03-27T08:26:50.677 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-27T08:26:53.692 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3015 ms) 2024-03-27T08:26:53.693 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-03-27T08:26:56.706 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RacingRemovePP (3013 ms) 2024-03-27T08:26:56.706 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-27T08:26:59.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (3008 ms) 2024-03-27T08:26:59.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-27T08:27:02.726 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3013 ms) 2024-03-27T08:27:02.726 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-03-27T08:27:05.229 INFO:tasks.workunit.client.0.smithi097.stdout:setting pool EIO 2024-03-27T08:27:14.900 INFO:tasks.workunit.client.0.smithi097.stdout:waiting for 765 2024-03-27T08:27:15.900 INFO:tasks.workunit.client.0.smithi097.stdout:waiting for 765 2024-03-27T08:27:16.900 INFO:tasks.workunit.client.0.smithi097.stdout:waiting for 765 2024-03-27T08:27:17.900 INFO:tasks.workunit.client.0.smithi097.stdout:max_success 235, min_failed 236 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAio.PoolEIOFlag (15305 ms) 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAio.MultiReads 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2427: Skipped 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 32 tests from LibRadosAio (122282 ms total) 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosAioPP 2024-03-27T08:27:18.032 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-27T08:27:21.046 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (3014 ms) 2024-03-27T08:27:21.046 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-27T08:27:29.701 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (8655 ms) 2024-03-27T08:27:29.701 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-03-27T08:27:44.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAioPP.RmXattrPP (14989 ms) 2024-03-27T08:27:44.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-27T08:27:47.697 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3007 ms) 2024-03-27T08:27:47.697 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosAioPP (29666 ms total) 2024-03-27T08:27:47.698 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:47.698 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosIoPP 2024-03-27T08:27:47.698 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-03-27T08:27:50.717 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.XattrListPP (3018 ms) 2024-03-27T08:27:50.717 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosIoPP (3019 ms total) 2024-03-27T08:27:50.717 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 20 tests from LibRadosAioEC 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1384: Skipped 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1419: Skipped 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1436: Skipped 2024-03-27T08:27:50.718 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1467: Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1500: Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1605: Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-03-27T08:27:50.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1637: Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1705: Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1742: Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1780: Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1796: Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1824: Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1859: Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1904: Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.721 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1946: Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:1977: Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2008: Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2059: Skipped 2024-03-27T08:27:50.722 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2089: Skipped 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/aio_cxx.cc:2124: Skipped 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 57 tests from 4 test suites ran. (154967 ms total) 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 36 tests. 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 21 tests, listed below: 2024-03-27T08:27:50.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-27T08:27:50.724 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:27:50.725 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:27:50.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_io 2024-03-27T08:27:50.737 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:27:50.737 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 24 tests from 2 test suites. 2024-03-27T08:27:50.737 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:27:50.737 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from LibRadosIo 2024-03-27T08:27:52.677 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-03-27T08:27:53.053 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.SimpleWrite (376 ms) 2024-03-27T08:27:53.053 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.TooBig 2024-03-27T08:27:53.053 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-03-27T08:27:53.053 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-03-27T08:27:53.115 INFO:tasks.workunit.client.0.smithi097.stdout:no timeout :/ 2024-03-27T08:27:53.116 INFO:tasks.workunit.client.0.smithi097.stdout:no timeout :/ 2024-03-27T08:27:53.116 INFO:tasks.workunit.client.0.smithi097.stdout:no timeout :/ 2024-03-27T08:27:53.116 INFO:tasks.workunit.client.0.smithi097.stdout:no timeout :/ 2024-03-27T08:27:53.116 INFO:tasks.workunit.client.0.smithi097.stdout:no timeout :/ 2024-03-27T08:27:53.116 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.ReadTimeout (62 ms) 2024-03-27T08:27:53.116 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-03-27T08:27:53.150 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.RoundTrip (35 ms) 2024-03-27T08:27:53.150 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.Checksum 2024-03-27T08:27:53.169 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.Checksum (19 ms) 2024-03-27T08:27:53.170 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-27T08:27:53.196 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (27 ms) 2024-03-27T08:27:53.197 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-27T08:27:53.223 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (27 ms) 2024-03-27T08:27:53.223 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-03-27T08:27:53.249 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.AppendRoundTrip (26 ms) 2024-03-27T08:27:53.249 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-03-27T08:27:53.254 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.ZeroLenZero (5 ms) 2024-03-27T08:27:53.254 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.TruncTest 2024-03-27T08:27:53.281 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.TruncTest (27 ms) 2024-03-27T08:27:53.281 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-03-27T08:27:53.308 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.RemoveTest (27 ms) 2024-03-27T08:27:53.308 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-27T08:27:53.338 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (30 ms) 2024-03-27T08:27:53.338 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.RmXattr 2024-03-27T08:27:53.414 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.RmXattr (76 ms) 2024-03-27T08:27:53.415 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIo.XattrIter 2024-03-27T08:27:53.450 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIo.XattrIter (36 ms) 2024-03-27T08:27:53.727 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from LibRadosIo (773 ms total) 2024-03-27T08:27:53.727 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 10 tests from LibRadosIoEC 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.728 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-03-27T08:27:53.729 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.730 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.731 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-03-27T08:27:53.732 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 24 tests from 2 test suites ran. (2989 ms total) 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 14 tests. 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 10 tests, listed below: 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-03-27T08:27:53.733 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-03-27T08:27:53.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-03-27T08:27:53.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-03-27T08:27:53.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-03-27T08:27:53.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:27:53.735 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:27:53.735 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_io_pp 2024-03-27T08:27:53.745 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:27:53.745 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 37 tests from 2 test suites. 2024-03-27T08:27:53.745 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:27:53.745 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 20 tests from LibRadosIoPP 2024-03-27T08:27:53.745 INFO:tasks.workunit.client.0.smithi097.stdout:seed 34452 2024-03-27T08:27:55.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-03-27T08:27:55.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-27T08:27:55.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-27T08:27:56.133 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (398 ms) 2024-03-27T08:27:56.133 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-03-27T08:27:56.203 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.ReadOpPP (70 ms) 2024-03-27T08:27:56.203 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-27T08:27:56.223 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (20 ms) 2024-03-27T08:27:56.223 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-03-27T08:27:56.238 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.RoundTripPP (15 ms) 2024-03-27T08:27:56.238 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-27T08:27:56.253 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (16 ms) 2024-03-27T08:27:56.253 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-03-27T08:27:56.268 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.Checksum (14 ms) 2024-03-27T08:27:56.268 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-27T08:27:56.284 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (16 ms) 2024-03-27T08:27:56.284 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-27T08:27:56.310 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (26 ms) 2024-03-27T08:27:56.310 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-27T08:27:56.335 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (25 ms) 2024-03-27T08:27:56.335 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-27T08:27:56.356 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (21 ms) 2024-03-27T08:27:56.356 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-27T08:27:56.387 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (31 ms) 2024-03-27T08:27:56.387 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-03-27T08:27:56.412 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.TruncTestPP (25 ms) 2024-03-27T08:27:56.412 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-27T08:27:56.438 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (26 ms) 2024-03-27T08:27:56.438 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-27T08:27:56.468 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (30 ms) 2024-03-27T08:27:56.468 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-03-27T08:27:56.541 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.RmXattrPP (73 ms) 2024-03-27T08:27:56.541 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-03-27T08:27:56.579 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.XattrListPP (38 ms) 2024-03-27T08:27:56.579 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-03-27T08:27:56.608 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.CmpExtPP (29 ms) 2024-03-27T08:27:56.608 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-27T08:27:56.624 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (16 ms) 2024-03-27T08:27:56.624 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-27T08:27:56.646 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (22 ms) 2024-03-27T08:27:56.749 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 20 tests from LibRadosIoPP (911 ms total) 2024-03-27T08:27:56.749 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:56.749 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 17 tests from LibRadosIoECPP 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.751 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-27T08:27:56.752 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.753 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.754 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-03-27T08:27:56.755 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.756 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (1 ms) 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 17 tests from LibRadosIoECPP (1 ms total) 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 37 tests from 2 test suites ran. (3001 ms total) 2024-03-27T08:27:56.757 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 20 tests. 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 17 tests, listed below: 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-27T08:27:56.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:27:56.759 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_asio 2024-03-27T08:27:56.774 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.782+0000 7fa3a9b446c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:27:56.779 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.787+0000 7fa3a9b446c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:27:56.779 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.787+0000 7fa3a9b446c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:27:56.780 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 12 tests from 1 test suite. 2024-03-27T08:27:56.780 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:27:56.780 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 12 tests from AsioRados 2024-03-27T08:27:56.780 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.788+0000 7fa3a9b446c0 1 Processor -- start 2024-03-27T08:27:56.780 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.788+0000 7fa3a9b446c0 1 -- start start 2024-03-27T08:27:56.783 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a9b446c0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:27:56.783 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a9b446c0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x69f0b0 con 0x721790 2024-03-27T08:27:56.783 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a259b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:27:56.783 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a259b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37230/0 (socket says 172.21.15.97:37230) 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a259b640 1 -- 172.21.15.97:0/613147919 learned_addr learned my addr 172.21.15.97:0/613147919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a259b640 1 -- 172.21.15.97:0/613147919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x63f1e0 con 0x721790 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a259b640 1 --2- 172.21.15.97:0/613147919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cf4a0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c46ea7dd67f9743 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a0d98640 1 -- 172.21.15.97:0/613147919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa38c0061e0 con 0x721790 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a0d98640 1 -- 172.21.15.97:0/613147919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa38c0019f0 con 0x721790 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.789+0000 7fa3a0d98640 1 -- 172.21.15.97:0/613147919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa38c006340 con 0x721790 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/613147919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 msgr2=0x7cf4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/613147919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cf4a0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/613147919 shutdown_connections 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/613147919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cf4a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/613147919 >> 172.21.15.97:0/613147919 conn(0x653ea0 msgr2=0x733400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/613147919 shutdown_connections 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.790+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/613147919 wait complete. 2024-03-27T08:27:56.784 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.791+0000 7fa3a9b446c0 1 Processor -- start 2024-03-27T08:27:56.785 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.791+0000 7fa3a9b446c0 1 -- start start 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a9b446c0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a9b446c0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x69cb30 con 0x721790 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a259b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a259b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37242/0 (socket says 172.21.15.97:37242) 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a259b640 1 -- 172.21.15.97:0/3160900898 learned_addr learned my addr 172.21.15.97:0/3160900898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a259b640 1 -- 172.21.15.97:0/3160900898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x631e30 con 0x721790 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a259b640 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cb1d0 crc :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa38c004900 con 0x721790 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa38c004a60 con 0x721790 2024-03-27T08:27:56.787 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x631890 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.792+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa38c004d00 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.793+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7355e0 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.793+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fa38c0143b0 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa39affd640 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa388036fe0 0x7fa3880394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(234..234 src has 1..234) ==== 2672+0+0 (crc 0 0 0) 0x7fa38c047f30 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x6931c0 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa3a1d9a640 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa388036fe0 0x7fa3880394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=234) ==== 24+0+0 (crc 0 0 0) 0x7fa38c010e90 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x6a4a30 con 0x721790 2024-03-27T08:27:56.788 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:56.794+0000 7fa3a1d9a640 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa388036fe0 0x7fa3880394f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:27:57.747 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.754+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v235) ==== 43+0+0 (crc 0 0 0) 0x7fa38c00f240 con 0x721790 2024-03-27T08:27:57.747 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.754+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=235}) -- 0x7fa38803b960 con 0x721790 2024-03-27T08:27:57.747 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.755+0000 7fa39affd640 1 -- 172.21.15.97:0/3160900898 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(235..235 src has 1..235) ==== 651+0+0 (crc 0 0 0) 0x7fa38c010c80 con 0x721790 2024-03-27T08:27:57.747 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.755+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> v2:172.21.15.97:6805/1668842645 conn(0x869cb0 0x86c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:27:57.747 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.755+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x85f120 con 0x869cb0 2024-03-27T08:27:57.748 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.757+0000 7fa3a2d9c640 1 --2- 172.21.15.97:0/3160900898 >> v2:172.21.15.97:6805/1668842645 conn(0x869cb0 0x86c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:27:57.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:57.765+0000 7fa3a2d9c640 1 --2- 172.21.15.97:0/3160900898 >> v2:172.21.15.97:6805/1668842645 conn(0x869cb0 0x86c0d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:27:58.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:58.992+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(2 exist [writefull 0~5] v236'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:58.984 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-03-27T08:27:58.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:58.992+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x866490 con 0x869cb0 2024-03-27T08:27:58.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:58.992+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x866cf0 con 0x869cb0 2024-03-27T08:27:59.004 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.012+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 2 ==== osd_op_reply(3 exist [read 0~5 out=5b] v236'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.011 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.019+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 3 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.011 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncReadCallback (27 ms) 2024-03-27T08:27:59.011 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-03-27T08:27:59.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.019+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x8670a0 con 0x869cb0 2024-03-27T08:27:59.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.019+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x86c610 con 0x869cb0 2024-03-27T08:27:59.036 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.044+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.037 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.044+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v236'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.037 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncReadFuture (25 ms) 2024-03-27T08:27:59.037 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-03-27T08:27:59.037 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.044+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x7cb710 con 0x869cb0 2024-03-27T08:27:59.037 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.044+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x8ad4b0 con 0x869cb0 2024-03-27T08:27:59.055 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.063+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 6 ==== osd_op_reply(7 exist [read 0~5 out=5b] v236'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.055 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.063+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 7 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.055 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncReadYield (19 ms) 2024-03-27T08:27:59.055 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-03-27T08:27:59.055 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.063+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x86d980 con 0x869cb0 2024-03-27T08:27:59.069 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.077+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 8 ==== osd_op_reply(9 exist [write 0~5] v236'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.069 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncWriteCallback (14 ms) 2024-03-27T08:27:59.069 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-03-27T08:27:59.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.077+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x86d980 con 0x869cb0 2024-03-27T08:27:59.079 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.087+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 9 ==== osd_op_reply(10 exist [write 0~5] v236'3 uv3 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.080 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncWriteFuture (11 ms) 2024-03-27T08:27:59.080 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-03-27T08:27:59.080 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.088+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x88feb0 con 0x869cb0 2024-03-27T08:27:59.090 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.098+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 10 ==== osd_op_reply(11 exist [write 0~5] v236'4 uv4 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.091 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncWriteYield (10 ms) 2024-03-27T08:27:59.091 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-27T08:27:59.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.098+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x86c970 con 0x869cb0 2024-03-27T08:27:59.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.098+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x86d2d0 con 0x869cb0 2024-03-27T08:27:59.097 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.105+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.097 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.105+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v236'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.097 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (7 ms) 2024-03-27T08:27:59.097 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-27T08:27:59.098 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.105+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x86ce20 con 0x869cb0 2024-03-27T08:27:59.098 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.105+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x8903c0 con 0x869cb0 2024-03-27T08:27:59.104 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.111+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 13 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.104 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.112+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v236'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.104 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (7 ms) 2024-03-27T08:27:59.104 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-03-27T08:27:59.105 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.112+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x86ce20 con 0x869cb0 2024-03-27T08:27:59.105 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.112+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) -- 0x86d550 con 0x869cb0 2024-03-27T08:27:59.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.120+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 15 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.120+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v236'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.113 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncReadOperationYield (8 ms) 2024-03-27T08:27:59.113 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-27T08:27:59.113 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.120+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x86ce20 con 0x869cb0 2024-03-27T08:27:59.125 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.133+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 17 ==== osd_op_reply(18 exist [writefull 0~5] v236'5 uv5 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.126 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (13 ms) 2024-03-27T08:27:59.126 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-27T08:27:59.126 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.133+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x86ce20 con 0x869cb0 2024-03-27T08:27:59.138 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.146+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 18 ==== osd_op_reply(19 exist [writefull 0~5] v236'6 uv6 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.138 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (13 ms) 2024-03-27T08:27:59.138 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-27T08:27:59.139 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.146+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) -- 0x86ce20 con 0x869cb0 2024-03-27T08:27:59.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.155+0000 7fa3a2d9c640 1 -- 172.21.15.97:0/3160900898 <== osd.1 v2:172.21.15.97:6805/1668842645 19 ==== osd_op_reply(20 exist [writefull 0~5] v236'7 uv7 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fa39400c430 con 0x869cb0 2024-03-27T08:27:59.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (10 ms) 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 >> v2:172.21.15.97:6805/1668842645 conn(0x869cb0 msgr2=0x86c0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> v2:172.21.15.97:6805/1668842645 conn(0x869cb0 0x86c0d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa388036fe0 msgr2=0x7fa3880394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa388036fe0 0x7fa3880394f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 msgr2=0x7cb1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cb1d0 crc :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 shutdown_connections 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> v2:172.21.15.97:6805/1668842645 conn(0x869cb0 0x86c0d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:59.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa388036fe0 0x7fa3880394f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 --2- 172.21.15.97:0/3160900898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x721790 0x7cb1d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 >> 172.21.15.97:0/3160900898 conn(0x653ea0 msgr2=0x733400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 shutdown_connections 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.156+0000 7fa3a9b446c0 1 -- 172.21.15.97:0/3160900898 wait complete. 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 12 tests from AsioRados (164 ms total) 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 12 tests from 1 test suite ran. (2369 ms total) 2024-03-27T08:27:59.150 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 12 tests. 2024-03-27T08:27:59.151 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:27:59.151 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:27:59.151 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_list 2024-03-27T08:27:59.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.187+0000 7f0813d199c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:27:59.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.187+0000 7f0813d199c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:27:59.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:27:59.187+0000 7f0813d199c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:27:59.180 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 11 tests from 3 test suites. 2024-03-27T08:27:59.180 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:27:59.180 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from LibRadosList 2024-03-27T08:28:00.762 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.ListObjects 2024-03-27T08:28:01.395 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.ListObjects (636 ms) 2024-03-27T08:28:01.395 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-27T08:28:01.599 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (204 ms) 2024-03-27T08:28:01.599 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-03-27T08:28:01.684 INFO:tasks.workunit.client.0.smithi097.stdout:myset foo1,foo2,foo3 2024-03-27T08:28:01.782 INFO:tasks.workunit.client.0.smithi097.stdout:foo1 2024-03-27T08:28:01.782 INFO:tasks.workunit.client.0.smithi097.stdout:foo2 2024-03-27T08:28:01.782 INFO:tasks.workunit.client.0.smithi097.stdout:foo3 2024-03-27T08:28:01.782 INFO:tasks.workunit.client.0.smithi097.stdout:myset foo1,foo4,foo5 2024-03-27T08:28:01.911 INFO:tasks.workunit.client.0.smithi097.stdout:foo4 2024-03-27T08:28:01.911 INFO:tasks.workunit.client.0.smithi097.stdout:foo5 2024-03-27T08:28:01.911 INFO:tasks.workunit.client.0.smithi097.stdout:foo1 2024-03-27T08:28:01.911 INFO:tasks.workunit.client.0.smithi097.stdout:myset foo6,foo7 2024-03-27T08:28:02.021 INFO:tasks.workunit.client.0.smithi097.stdout:foo7 2024-03-27T08:28:02.021 INFO:tasks.workunit.client.0.smithi097.stdout:foo6 2024-03-27T08:28:02.021 INFO:tasks.workunit.client.0.smithi097.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout:ns1:foo4 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout:ns1:foo5 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout:ns2:foo7 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout:ns2:foo6 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout:ns1:foo1 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout::foo1 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout::foo2 2024-03-27T08:28:02.115 INFO:tasks.workunit.client.0.smithi097.stdout::foo3 2024-03-27T08:28:02.285 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.ListObjectsNS (686 ms) 2024-03-27T08:28:02.286 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:1 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:10 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:13 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:7 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:14 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:0 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:15 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:11 0 2024-03-27T08:28:02.550 INFO:tasks.workunit.client.0.smithi097.stdout:5 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:8 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:6 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:3 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:4 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:12 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:9 0 2024-03-27T08:28:02.551 INFO:tasks.workunit.client.0.smithi097.stdout:2 0 2024-03-27T08:28:02.644 INFO:tasks.workunit.client.0.smithi097.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-27T08:28:03.084 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.ListObjectsStart (798 ms) 2024-03-27T08:28:03.084 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-03-27T08:28:03.335 INFO:tasks.workunit.client.0.smithi097.stdout:x cursor=MIN 2024-03-27T08:28:03.428 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-03-27T08:28:03.429 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-03-27T08:28:03.430 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-03-27T08:28:03.430 INFO:tasks.workunit.client.0.smithi097.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-03-27T08:28:03.523 INFO:tasks.workunit.client.0.smithi097.stdout:FIRST> seek to MIN oid=1 2024-03-27T08:28:03.621 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-03-27T08:28:03.622 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:02547ec2:::1:head 2024-03-27T08:28:03.716 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:02547ec2:::1:head -> 1 2024-03-27T08:28:03.716 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-03-27T08:28:03.716 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:52ea6a34:::10:head 2024-03-27T08:28:03.780 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:52ea6a34:::10:head -> 10 2024-03-27T08:28:03.780 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-03-27T08:28:03.780 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:566253c9:::13:head 2024-03-27T08:28:03.844 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:566253c9:::13:head -> 13 2024-03-27T08:28:03.845 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-03-27T08:28:03.845 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:5c6b0b28:::7:head 2024-03-27T08:28:03.905 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:5c6b0b28:::7:head -> 7 2024-03-27T08:28:03.906 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-03-27T08:28:03.906 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:62a1935d:::14:head 2024-03-27T08:28:03.963 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:62a1935d:::14:head -> 14 2024-03-27T08:28:03.964 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-03-27T08:28:03.964 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:6cac518f:::0:head 2024-03-27T08:28:04.020 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:6cac518f:::0:head -> 0 2024-03-27T08:28:04.020 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-03-27T08:28:04.020 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:863748b0:::15:head 2024-03-27T08:28:04.067 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:863748b0:::15:head -> 15 2024-03-27T08:28:04.068 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-03-27T08:28:04.068 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:89d3ae78:::11:head 2024-03-27T08:28:04.110 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:89d3ae78:::11:head -> 11 2024-03-27T08:28:04.111 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-03-27T08:28:04.111 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:b29083e3:::5:head 2024-03-27T08:28:04.139 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:b29083e3:::5:head -> 5 2024-03-27T08:28:04.139 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-03-27T08:28:04.140 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:bd63b0f1:::8:head 2024-03-27T08:28:04.165 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:bd63b0f1:::8:head -> 8 2024-03-27T08:28:04.166 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-03-27T08:28:04.166 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:c4fdafeb:::6:head 2024-03-27T08:28:04.188 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:c4fdafeb:::6:head -> 6 2024-03-27T08:28:04.189 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-03-27T08:28:04.189 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:cfc208b3:::3:head 2024-03-27T08:28:04.209 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:cfc208b3:::3:head -> 3 2024-03-27T08:28:04.209 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-03-27T08:28:04.209 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:d83876eb:::4:head 2024-03-27T08:28:04.223 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:d83876eb:::4:head -> 4 2024-03-27T08:28:04.223 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-03-27T08:28:04.223 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:de5d7c5f:::12:head 2024-03-27T08:28:04.237 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:de5d7c5f:::12:head -> 12 2024-03-27T08:28:04.237 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-03-27T08:28:04.237 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:e960b815:::9:head 2024-03-27T08:28:04.246 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:e960b815:::9:head -> 9 2024-03-27T08:28:04.246 INFO:tasks.workunit.client.0.smithi097.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-03-27T08:28:04.246 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:f905c69b:::2:head 2024-03-27T08:28:04.249 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:f905c69b:::2:head -> 2 2024-03-27T08:28:04.249 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:bd63b0f1:::8:head 2024-03-27T08:28:04.249 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-03-27T08:28:04.275 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:bd63b0f1:::8:head -> 8 2024-03-27T08:28:04.275 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=8 expected=8 2024-03-27T08:28:04.275 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:5c6b0b28:::7:head 2024-03-27T08:28:04.276 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-03-27T08:28:04.337 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:5c6b0b28:::7:head -> 7 2024-03-27T08:28:04.337 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=7 expected=7 2024-03-27T08:28:04.337 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:566253c9:::13:head 2024-03-27T08:28:04.338 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-03-27T08:28:04.402 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:566253c9:::13:head -> 13 2024-03-27T08:28:04.402 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=13 expected=13 2024-03-27T08:28:04.402 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:c4fdafeb:::6:head 2024-03-27T08:28:04.403 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-03-27T08:28:04.425 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:c4fdafeb:::6:head -> 6 2024-03-27T08:28:04.425 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=6 expected=6 2024-03-27T08:28:04.425 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:b29083e3:::5:head 2024-03-27T08:28:04.425 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-03-27T08:28:04.454 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:b29083e3:::5:head -> 5 2024-03-27T08:28:04.454 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=5 expected=5 2024-03-27T08:28:04.454 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:52ea6a34:::10:head 2024-03-27T08:28:04.455 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-03-27T08:28:04.519 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:52ea6a34:::10:head -> 10 2024-03-27T08:28:04.520 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=10 expected=10 2024-03-27T08:28:04.520 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:89d3ae78:::11:head 2024-03-27T08:28:04.520 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-03-27T08:28:04.562 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:89d3ae78:::11:head -> 11 2024-03-27T08:28:04.563 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=11 expected=11 2024-03-27T08:28:04.563 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:863748b0:::15:head 2024-03-27T08:28:04.563 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-03-27T08:28:04.610 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:863748b0:::15:head -> 15 2024-03-27T08:28:04.610 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=15 expected=15 2024-03-27T08:28:04.610 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:6cac518f:::0:head 2024-03-27T08:28:04.610 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-03-27T08:28:04.666 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:6cac518f:::0:head -> 0 2024-03-27T08:28:04.666 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=0 expected=0 2024-03-27T08:28:04.666 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:62a1935d:::14:head 2024-03-27T08:28:04.666 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-03-27T08:28:04.725 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:62a1935d:::14:head -> 14 2024-03-27T08:28:04.725 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=14 expected=14 2024-03-27T08:28:04.725 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:f905c69b:::2:head 2024-03-27T08:28:04.725 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-03-27T08:28:04.728 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:f905c69b:::2:head -> 2 2024-03-27T08:28:04.728 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=2 expected=2 2024-03-27T08:28:04.728 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:e960b815:::9:head 2024-03-27T08:28:04.728 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-03-27T08:28:04.737 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:e960b815:::9:head -> 9 2024-03-27T08:28:04.737 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=9 expected=9 2024-03-27T08:28:04.737 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:de5d7c5f:::12:head 2024-03-27T08:28:04.737 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-03-27T08:28:04.751 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:de5d7c5f:::12:head -> 12 2024-03-27T08:28:04.751 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=12 expected=12 2024-03-27T08:28:04.751 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:d83876eb:::4:head 2024-03-27T08:28:04.751 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-03-27T08:28:04.766 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:d83876eb:::4:head -> 4 2024-03-27T08:28:04.766 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=4 expected=4 2024-03-27T08:28:04.766 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:cfc208b3:::3:head 2024-03-27T08:28:04.766 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-03-27T08:28:04.785 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:cfc208b3:::3:head -> 3 2024-03-27T08:28:04.786 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=3 expected=3 2024-03-27T08:28:04.786 INFO:tasks.workunit.client.0.smithi097.stdout:: seek to 76:02547ec2:::1:head 2024-03-27T08:28:04.786 INFO:tasks.workunit.client.0.smithi097.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-03-27T08:28:04.881 INFO:tasks.workunit.client.0.smithi097.stdout:> 76:02547ec2:::1:head -> 1 2024-03-27T08:28:04.881 INFO:tasks.workunit.client.0.smithi097.stdout:: entry=1 expected=1 2024-03-27T08:28:05.146 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.ListObjectsCursor (2063 ms) 2024-03-27T08:28:05.146 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-03-27T08:28:05.669 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.EnumerateObjects (523 ms) 2024-03-27T08:28:05.669 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-27T08:28:05.835 INFO:tasks.workunit.client.0.smithi097.stdout:split 0/5 -> MIN 76:33333333::::head 2024-03-27T08:28:05.856 INFO:tasks.workunit.client.0.smithi097.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-03-27T08:28:05.878 INFO:tasks.workunit.client.0.smithi097.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-03-27T08:28:05.902 INFO:tasks.workunit.client.0.smithi097.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-03-27T08:28:05.922 INFO:tasks.workunit.client.0.smithi097.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-03-27T08:28:06.199 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (530 ms) 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from LibRadosList (5440 ms total) 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosListEC 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:70: Skipped 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:84: Skipped 2024-03-27T08:28:07.041 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:96: Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:84: Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:96: Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:84: Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:96: Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.042 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-03-27T08:28:07.043 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:78: Skipped 2024-03-27T08:28:07.043 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:07.043 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-03-27T08:28:07.043 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:07.043 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosListNP 2024-03-27T08:28:07.043 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-03-27T08:28:09.689 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosListNP.ListObjectsError (2649 ms) 2024-03-27T08:28:09.689 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosListNP (2649 ms total) 2024-03-27T08:28:09.689 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:09.689 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:28:09.690 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 11 tests from 3 test suites ran. (10510 ms total) 2024-03-27T08:28:09.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 8 tests. 2024-03-27T08:28:09.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 3 tests, listed below: 2024-03-27T08:28:09.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-03-27T08:28:09.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-03-27T08:28:09.690 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-03-27T08:28:09.692 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:28:09.692 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:28:09.692 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_lock 2024-03-27T08:28:09.710 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:28:09.710 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-27T08:28:09.710 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:28:09.710 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLock 2024-03-27T08:28:11.695 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-03-27T08:28:12.023 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.LockExclusive (328 ms) 2024-03-27T08:28:12.024 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.LockShared 2024-03-27T08:28:12.060 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.LockShared (37 ms) 2024-03-27T08:28:12.060 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-03-27T08:28:13.097 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.LockExclusiveDur (1037 ms) 2024-03-27T08:28:13.097 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-03-27T08:28:14.143 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.LockSharedDur (1046 ms) 2024-03-27T08:28:14.144 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-03-27T08:28:14.181 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.LockMayRenew (38 ms) 2024-03-27T08:28:14.181 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.Unlock 2024-03-27T08:28:14.221 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.Unlock (40 ms) 2024-03-27T08:28:14.221 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.ListLockers 2024-03-27T08:28:14.275 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.ListLockers (54 ms) 2024-03-27T08:28:14.275 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLock.BreakLock 2024-03-27T08:28:14.310 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLock.BreakLock (35 ms) 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLock (2615 ms total) 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLockEC 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.711 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-03-27T08:28:14.712 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.713 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:28:14.714 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLockEC (1 ms total) 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 16 tests from 2 test suites ran. (5000 ms total) 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 8 tests. 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 8 tests, listed below: 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-03-27T08:28:14.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-03-27T08:28:14.716 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-03-27T08:28:14.716 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-03-27T08:28:14.716 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-03-27T08:28:14.716 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-03-27T08:28:14.716 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-03-27T08:28:14.717 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:28:14.717 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:28:14.717 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_lock_pp 2024-03-27T08:28:14.734 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:28:14.734 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-27T08:28:14.734 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:28:14.735 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLockPP 2024-03-27T08:28:14.735 INFO:tasks.workunit.client.0.smithi097.stdout:seed 34567 2024-03-27T08:28:16.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-27T08:28:17.095 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (371 ms) 2024-03-27T08:28:17.095 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-03-27T08:28:17.115 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.LockSharedPP (21 ms) 2024-03-27T08:28:17.115 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-27T08:28:18.149 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (1034 ms) 2024-03-27T08:28:18.149 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-27T08:28:19.187 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1037 ms) 2024-03-27T08:28:19.187 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-27T08:28:19.226 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (40 ms) 2024-03-27T08:28:19.226 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-03-27T08:28:19.299 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.UnlockPP (73 ms) 2024-03-27T08:28:19.299 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-03-27T08:28:19.349 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.ListLockersPP (50 ms) 2024-03-27T08:28:19.349 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-03-27T08:28:19.379 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosLockPP.BreakLockPP (29 ms) 2024-03-27T08:28:19.737 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLockPP (2655 ms total) 2024-03-27T08:28:19.737 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:19.737 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLockECPP 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-27T08:28:19.738 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (0 ms) 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.739 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.740 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (1 ms) 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.741 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosLockECPP (1 ms total) 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 16 tests from 2 test suites ran. (5000 ms total) 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 8 tests. 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 8 tests, listed below: 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-27T08:28:19.742 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:28:19.743 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_misc 2024-03-27T08:28:19.766 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:19.774+0000 7f3c57a786c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:28:19.767 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:19.774+0000 7f3c57a786c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:28:19.767 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:19.774+0000 7f3c57a786c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:28:19.767 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 12 tests from 4 test suites. 2024-03-27T08:28:19.767 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:28:19.767 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscVersion 2024-03-27T08:28:19.768 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-03-27T08:28:19.768 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-27T08:28:19.768 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscVersion (1 ms total) 2024-03-27T08:28:19.768 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:19.768 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-03-27T08:28:19.768 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-27T08:28:19.798 INFO:tasks.workunit.client.0.smithi097.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-27T08:28:19.798 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:19.806+0000 7f3c57a786c0 -1 failed for service _ceph-mon._tcp 2024-03-27T08:28:19.798 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:19.806+0000 7f3c57a786c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-27T08:28:19.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (31 ms) 2024-03-27T08:28:19.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-27T08:28:21.805 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2006 ms) 2024-03-27T08:28:21.805 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (2037 ms total) 2024-03-27T08:28:21.805 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:21.805 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscPool 2024-03-27T08:28:21.805 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-27T08:28:23.686 INFO:tasks.workunit.client.0.smithi097.stdout:started 0x7f3c20005d20 2024-03-27T08:28:24.686 INFO:tasks.workunit.client.0.smithi097.stdout:started 0x7f3c4000a1c0 2024-03-27T08:28:24.686 INFO:tasks.workunit.client.0.smithi097.stdout: started 2 aios 2024-03-27T08:28:24.686 INFO:tasks.workunit.client.0.smithi097.stdout:waiting 0x7f3c20005d20 2024-03-27T08:28:24.686 INFO:tasks.workunit.client.0.smithi097.stdout:waiting 0x7f3c4000a1c0 2024-03-27T08:28:24.689 INFO:tasks.workunit.client.0.smithi097.stdout:done. 2024-03-27T08:28:26.703 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (4899 ms) 2024-03-27T08:28:26.703 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscPool (4899 ms total) 2024-03-27T08:28:26.703 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:26.703 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosMisc 2024-03-27T08:28:28.708 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-03-27T08:28:28.708 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-27T08:28:28.708 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.Exec 2024-03-27T08:28:29.078 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.Exec (370 ms) 2024-03-27T08:28:29.078 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-03-27T08:28:29.120 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.WriteSame (42 ms) 2024-03-27T08:28:29.120 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-03-27T08:28:29.145 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.CmpExt (25 ms) 2024-03-27T08:28:29.145 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.Applications 2024-03-27T08:28:33.687 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.Applications (4542 ms) 2024-03-27T08:28:33.687 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-03-27T08:28:33.687 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-27T08:28:33.688 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-03-27T08:28:33.688 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-27T08:28:33.688 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-03-27T08:28:59.066 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMisc.ShutdownRace (25378 ms) 2024-03-27T08:28:59.705 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 8 tests from LibRadosMisc (30357 ms total) 2024-03-27T08:28:59.705 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:59.705 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:28:59.705 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 12 tests from 4 test suites ran. (39939 ms total) 2024-03-27T08:28:59.705 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 12 tests. 2024-03-27T08:28:59.724 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:28:59.724 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:28:59.724 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_misc_pp 2024-03-27T08:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:59.766+0000 7fdd8c229840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:28:59.759 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:59.766+0000 7fdd8c229840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:28:59.759 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:28:59.766+0000 7fdd8c229840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:28:59.760 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 31 tests from 7 test suites. 2024-03-27T08:28:59.760 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:28:59.760 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscVersion 2024-03-27T08:28:59.760 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-03-27T08:28:59.760 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-27T08:28:59.760 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-27T08:28:59.761 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:28:59.761 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 22 tests from LibRadosMiscPP 2024-03-27T08:28:59.761 INFO:tasks.workunit.client.0.smithi097.stdout:seed 96233 2024-03-27T08:29:01.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-27T08:29:01.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-03-27T08:29:01.719 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-03-27T08:29:02.217 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.LongNamePP (497 ms) 2024-03-27T08:29:02.217 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-27T08:29:02.360 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.LongLocatorPP (143 ms) 2024-03-27T08:29:02.361 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-27T08:29:02.413 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (53 ms) 2024-03-27T08:29:02.413 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-27T08:29:02.457 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (44 ms) 2024-03-27T08:29:02.457 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-03-27T08:29:02.473 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.ExecPP (16 ms) 2024-03-27T08:29:02.473 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-27T08:29:02.488 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (15 ms) 2024-03-27T08:29:02.488 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-03-27T08:29:02.514 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.Operate1PP (26 ms) 2024-03-27T08:29:02.514 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-03-27T08:29:02.530 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.Operate2PP (16 ms) 2024-03-27T08:29:02.530 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-27T08:29:02.577 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (47 ms) 2024-03-27T08:29:02.577 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-27T08:29:02.594 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (17 ms) 2024-03-27T08:29:02.594 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-27T08:29:02.618 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (24 ms) 2024-03-27T08:29:02.618 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-27T08:29:02.665 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (47 ms) 2024-03-27T08:29:02.665 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-27T08:29:02.677 INFO:tasks.workunit.client.0.smithi097.stdout:osd_max_attr_size = 0 2024-03-27T08:29:02.677 INFO:tasks.workunit.client.0.smithi097.stdout:osd_max_attr_size == 0; skipping test 2024-03-27T08:29:21.164 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (18499 ms) 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:386: Skipped 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (0 ms) 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:549: Skipped 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-03-27T08:29:21.165 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-27T08:29:21.201 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (37 ms) 2024-03-27T08:29:21.201 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-27T08:29:21.222 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (21 ms) 2024-03-27T08:29:21.222 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-03-27T08:29:25.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.Applications (4478 ms) 2024-03-27T08:29:25.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-27T08:29:25.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-27T08:29:25.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-27T08:29:25.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-27T08:29:25.700 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-03-27T08:29:25.701 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-27T08:29:26.708 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 22 tests from LibRadosMiscPP (23981 ms total) 2024-03-27T08:29:26.708 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:26.708 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-03-27T08:29:26.708 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:476: Skipped 2024-03-27T08:29:26.708 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:498: Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:504: Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/misc_cxx.cc:491: Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:26.709 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-27T08:29:28.718 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-03-27T08:29:29.058 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosChecksum/0.Subset (339 ms) 2024-03-27T08:29:29.058 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-03-27T08:29:29.118 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosChecksum/0.Chunked (61 ms) 2024-03-27T08:29:29.722 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosChecksum/0 (400 ms total) 2024-03-27T08:29:29.722 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:29.723 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-27T08:29:31.730 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-03-27T08:29:32.067 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosChecksum/1.Subset (337 ms) 2024-03-27T08:29:32.067 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-03-27T08:29:32.140 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosChecksum/1.Chunked (73 ms) 2024-03-27T08:29:32.697 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosChecksum/1 (410 ms total) 2024-03-27T08:29:32.698 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:32.698 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-27T08:29:34.699 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-03-27T08:29:35.066 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosChecksum/2.Subset (367 ms) 2024-03-27T08:29:35.066 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-03-27T08:29:35.089 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosChecksum/2.Chunked (23 ms) 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosChecksum/2 (390 ms total) 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscECPP 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange (0 ms) 2024-03-27T08:29:35.743 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosMiscECPP (0 ms total) 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 31 tests from 7 test suites ran. (35983 ms total) 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 27 tests. 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-27T08:29:35.744 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange 2024-03-27T08:29:35.745 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:29:35.745 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:29:35.745 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_tier_pp 2024-03-27T08:29:35.775 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:29:35.783+0000 7f209ab3d9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:29:35.775 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:29:35.783+0000 7f209ab3d9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:29:35.775 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:29:35.783+0000 7f209ab3d9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:29:35.776 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 77 tests from 4 test suites. 2024-03-27T08:29:35.776 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:29:35.776 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosTierPP 2024-03-27T08:29:35.776 INFO:tasks.workunit.client.0.smithi097.stdout:seed 96305 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierPP.Dirty 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:287: Skipped 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:2182: Skipped 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierPP.HitSetNone 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:2561: Skipped 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:37.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone (0 ms) 2024-03-27T08:29:38.750 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosTierPP (0 ms total) 2024-03-27T08:29:38.750 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:38.750 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 48 tests from LibRadosTwoPoolsPP 2024-03-27T08:29:40.778 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-27T08:29:40.778 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.778 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.778 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (0 ms) 2024-03-27T08:29:40.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush (0 ms) 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2024-03-27T08:29:40.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-27T08:29:40.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (0 ms) 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2024-03-27T08:29:40.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (0 ms) 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2024-03-27T08:29:40.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (0 ms) 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-27T08:29:40.793 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.794 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2024-03-27T08:29:40.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (1 ms) 2024-03-27T08:29:40.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.797 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.798 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:40.799 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (0 ms) 2024-03-27T08:29:41.778 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 48 tests from LibRadosTwoPoolsPP (1 ms total) 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosTierECPP 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierECPP.Dirty 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:29:41.779 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTierECPP.HitSetNone 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:29:41.780 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosTierECPP (0 ms total) 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6267: Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.781 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (1 ms) 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (0 ms) 2024-03-27T08:29:41.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.783 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.784 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2024-03-27T08:29:41.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-27T08:29:41.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (0 ms) 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-27T08:29:41.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-27T08:29:41.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/tier_cxx.cc:6272: Skipped 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP (1 ms total) 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 77 tests from 4 test suites ran. (5998 ms total) 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 0 tests. 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 77 tests, listed below: 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierPP.Dirty 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2024-03-27T08:29:41.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush 2024-03-27T08:29:41.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-27T08:29:41.793 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-27T08:29:41.794 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-27T08:29:41.795 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-27T08:29:41.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-27T08:29:41.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-27T08:29:41.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-27T08:29:41.797 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:29:41.797 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:29:41.797 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_pool 2024-03-27T08:29:41.798 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:29:41.798 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-27T08:29:41.798 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:29:41.798 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 9 tests from LibRadosPools 2024-03-27T08:29:41.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolList 2024-03-27T08:29:44.705 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolList (2910 ms) 2024-03-27T08:29:44.706 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolLookup 2024-03-27T08:29:47.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolLookup (3015 ms) 2024-03-27T08:29:47.720 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolLookup2 2024-03-27T08:29:50.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolLookup2 (3015 ms) 2024-03-27T08:29:50.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-27T08:29:53.707 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolLookupOtherInstance (2972 ms) 2024-03-27T08:29:53.707 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-27T08:29:56.772 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3065 ms) 2024-03-27T08:29:56.773 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolDelete 2024-03-27T08:30:01.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolDelete (5019 ms) 2024-03-27T08:30:01.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolCreateDelete 2024-03-27T08:30:06.714 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolCreateDelete (4923 ms) 2024-03-27T08:30:06.715 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPools.PoolCreateWithCrushRule (5033 ms) 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/pool.cc:139: Skipped 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 9 tests from LibRadosPools (29952 ms total) 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 9 tests from 1 test suite ran. (29952 ms total) 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 8 tests. 2024-03-27T08:30:11.748 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-27T08:30:11.749 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier 2024-03-27T08:30:11.750 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:30:11.750 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:30:11.750 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_snapshots 2024-03-27T08:30:11.771 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:30:11.771 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 13 tests from 4 test suites. 2024-03-27T08:30:11.771 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:30:11.771 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosSnapshots 2024-03-27T08:30:13.706 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshots.SnapList 2024-03-27T08:30:15.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshots.SnapList (2052 ms) 2024-03-27T08:30:15.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshots.SnapRemove 2024-03-27T08:30:17.764 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshots.SnapRemove (2006 ms) 2024-03-27T08:30:17.764 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshots.Rollback 2024-03-27T08:30:19.771 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshots.Rollback (2007 ms) 2024-03-27T08:30:19.771 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshots.SnapGetName 2024-03-27T08:30:21.782 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshots.SnapGetName (2011 ms) 2024-03-27T08:30:22.712 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosSnapshots (8076 ms total) 2024-03-27T08:30:22.712 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:30:22.712 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-27T08:30:24.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-27T08:30:28.909 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Snap (4196 ms) 2024-03-27T08:30:28.909 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-27T08:30:32.849 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Rollback (3939 ms) 2024-03-27T08:30:32.849 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-27T08:30:37.892 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5044 ms) 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged (13179 ms total) 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosSnapshotsEC 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:38.786 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList (0 ms) 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove (0 ms) 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:38.787 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback (0 ms) 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName (0 ms) 2024-03-27T08:30:38.788 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosSnapshotsEC (0 ms total) 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap (0 ms) 2024-03-27T08:30:38.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback (0 ms) 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC (0 ms total) 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:30:38.790 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 13 tests from 4 test suites ran. (27012 ms total) 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 7 tests. 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 6 tests, listed below: 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-27T08:30:38.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-27T08:30:38.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:30:38.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:30:38.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_snapshots_pp 2024-03-27T08:30:38.808 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:30:38.808 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 21 tests from 5 test suites. 2024-03-27T08:30:38.808 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:30:38.808 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from LibRadosSnapshotsPP 2024-03-27T08:30:38.808 INFO:tasks.workunit.client.0.smithi097.stdout:seed 96566 2024-03-27T08:30:40.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-27T08:30:42.711 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsPP.SnapListPP (1919 ms) 2024-03-27T08:30:42.711 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-27T08:30:44.806 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsPP.SnapRemovePP (2095 ms) 2024-03-27T08:30:44.806 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-27T08:30:46.815 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsPP.RollbackPP (2009 ms) 2024-03-27T08:30:46.815 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-27T08:30:48.830 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2015 ms) 2024-03-27T08:30:48.830 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-27T08:30:52.713 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3883 ms) 2024-03-27T08:30:53.715 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from LibRadosSnapshotsPP (11921 ms total) 2024-03-27T08:30:53.715 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:30:53.715 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-27T08:30:55.723 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4126 ms) 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/snapshots_cxx.cc:156: Skipped 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP (0 ms) 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/snapshots_cxx.cc:242: Skipped 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (0 ms) 2024-03-27T08:30:59.850 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-27T08:31:03.716 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3867 ms) 2024-03-27T08:31:03.717 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-27T08:31:05.886 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2170 ms) 2024-03-27T08:31:05.886 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-27T08:31:05.886 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-27T08:31:05.887 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:05.887 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-27T08:31:05.887 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-27T08:31:07.764 INFO:tasks.workunit.client.0.smithi097.stdout:deleting snap 9 in pool LibRadosSnapshotsSelfManagedPP_smithi097-96566-7 2024-03-27T08:31:08.770 INFO:tasks.workunit.client.0.smithi097.stdout:waiting for snaps to purge 2024-03-27T08:31:23.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17901 ms) 2024-03-27T08:31:24.090 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP (28064 ms total) 2024-03-27T08:31:24.090 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:24.090 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-27T08:31:24.090 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-27T08:31:24.096 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (6 ms) 2024-03-27T08:31:24.096 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-27T08:31:29.156 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5057 ms) 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5063 ms total) 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosSnapshotsECPP 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.157 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP (1 ms) 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP (0 ms) 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP (0 ms) 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP (0 ms) 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosSnapshotsECPP (1 ms total) 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:31:29.159 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP (0 ms) 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (0 ms) 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:29.160 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (0 ms) 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (0 ms total) 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 21 tests from 5 test suites ran. (50346 ms total) 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 11 tests. 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 10 tests, listed below: 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-27T08:31:29.161 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:29.162 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_stat 2024-03-27T08:31:29.176 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:31:29.176 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 9 tests from 2 test suites. 2024-03-27T08:31:29.176 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:29.176 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from LibRadosStat 2024-03-27T08:31:31.122 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStat.Stat 2024-03-27T08:31:31.461 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStat.Stat (339 ms) 2024-03-27T08:31:31.461 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStat.Stat2 2024-03-27T08:31:31.493 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStat.Stat2 (33 ms) 2024-03-27T08:31:31.493 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStat.StatNS 2024-03-27T08:31:31.544 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStat.StatNS (50 ms) 2024-03-27T08:31:31.544 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStat.ClusterStat 2024-03-27T08:31:31.544 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-27T08:31:31.545 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStat.PoolStat 2024-03-27T08:31:31.555 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStat.PoolStat (12 ms) 2024-03-27T08:31:32.133 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from LibRadosStat (434 ms total) 2024-03-27T08:31:32.145 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosStatEC 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatEC.Stat 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.Stat (0 ms) 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatEC.StatNS 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.146 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.StatNS (0 ms) 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatEC.ClusterStat 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat (0 ms) 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatEC.PoolStat 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:32.147 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat (0 ms) 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosStatEC (0 ms total) 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 9 tests from 2 test suites ran. (2956 ms total) 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 5 tests. 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.Stat 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.StatNS 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat 2024-03-27T08:31:32.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat 2024-03-27T08:31:32.149 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:32.149 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:32.149 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_stat_pp 2024-03-27T08:31:32.153 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:31:32.153 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 9 tests from 2 test suites. 2024-03-27T08:31:32.153 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:32.153 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from LibRadosStatPP 2024-03-27T08:31:32.153 INFO:tasks.workunit.client.0.smithi097.stdout:seed 96701 2024-03-27T08:31:33.725 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatPP.StatPP 2024-03-27T08:31:34.098 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStatPP.StatPP (370 ms) 2024-03-27T08:31:34.098 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-27T08:31:34.119 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStatPP.Stat2Mtime2PP (24 ms) 2024-03-27T08:31:34.119 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-27T08:31:34.119 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-03-27T08:31:34.119 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatPP.PoolStatPP 2024-03-27T08:31:34.135 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStatPP.PoolStatPP (17 ms) 2024-03-27T08:31:34.135 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatPP.StatPPNS 2024-03-27T08:31:34.191 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosStatPP.StatPPNS (55 ms) 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from LibRadosStatPP (466 ms total) 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosStatECPP 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatECPP.StatPP 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.734 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP (0 ms) 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.735 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP (0 ms) 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosStatECPP.StatPPNS 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS (0 ms) 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosStatECPP (0 ms total) 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 9 tests from 2 test suites ran. (2581 ms total) 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 5 tests. 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP 2024-03-27T08:31:34.736 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP 2024-03-27T08:31:34.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP 2024-03-27T08:31:34.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS 2024-03-27T08:31:34.737 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:34.737 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:34.737 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_watch_notify 2024-03-27T08:31:34.754 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:31:34.754 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 11 tests from 2 test suites. 2024-03-27T08:31:34.754 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:34.754 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 10 tests from LibRadosWatchNotify 2024-03-27T08:31:36.737 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-27T08:31:37.113 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify_test_cb 2024-03-27T08:31:37.141 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.WatchNotify (405 ms) 2024-03-27T08:31:37.141 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-27T08:31:37.172 INFO:tasks.workunit.client.0.smithi097.stdout:waiting up to 300 for disconnect notification ... 2024-03-27T08:31:39.787 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_errcb cookie 15238432 err -107 2024-03-27T08:31:40.182 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.Watch2Delete (3040 ms) 2024-03-27T08:31:40.182 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-27T08:31:40.226 INFO:tasks.workunit.client.0.smithi097.stdout:waiting up to 300 for disconnect notification ... 2024-03-27T08:31:44.789 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_errcb cookie 15250096 err -107 2024-03-27T08:31:45.235 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete (5053 ms) 2024-03-27T08:31:45.235 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-27T08:31:45.265 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114624 cookie 15255072 2024-03-27T08:31:45.286 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2 (51 ms) 2024-03-27T08:31:45.286 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-27T08:31:45.314 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114625 cookie 15274240 2024-03-27T08:31:45.334 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.AioWatchNotify2 (48 ms) 2024-03-27T08:31:45.334 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.AioNotify 2024-03-27T08:31:45.360 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114626 cookie 15274240 2024-03-27T08:31:45.380 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.AioNotify (46 ms) 2024-03-27T08:31:45.380 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-27T08:31:45.416 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114624 cookie 15274240 2024-03-27T08:31:45.416 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114624 cookie 15275520 2024-03-27T08:31:45.444 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Multi (64 ms) 2024-03-27T08:31:45.445 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-27T08:31:45.470 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114625 cookie 15275520 2024-03-27T08:31:48.470 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114626 cookie 15275520 2024-03-27T08:31:48.493 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3049 ms) 2024-03-27T08:31:48.493 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-27T08:31:48.519 INFO:tasks.workunit.client.0.smithi097.stdout:waiting up to 1024 for osd to time us out ... 2024-03-27T08:31:52.528 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_errcb cookie 15271840 err -107 2024-03-27T08:31:53.549 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_cb from 15603 notify_id 1692217114626 cookie 15271840 2024-03-27T08:31:53.566 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.Watch3Timeout (5073 ms) 2024-03-27T08:31:53.566 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-27T08:31:53.602 INFO:tasks.workunit.client.0.smithi097.stdout:waiting up to 30 for disconnect notification ... 2024-03-27T08:31:54.790 INFO:tasks.workunit.client.0.smithi097.stdout:watch_notify2_test_errcb cookie 15271840 err -107 2024-03-27T08:31:55.611 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete2 (2045 ms) 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 10 tests from LibRadosWatchNotify (18874 ms total) 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosWatchNotifyEC 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:168: Skipped 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:182: Skipped 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:196: Skipped 2024-03-27T08:31:55.766 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify (0 ms) 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/TestCase.cc:176: Skipped 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibRadosWatchNotifyEC (0 ms total) 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 11 tests from 2 test suites ran. (21011 ms total) 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 10 tests. 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-27T08:31:55.767 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify 2024-03-27T08:31:55.768 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:31:55.768 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:31:55.768 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-03-27T08:31:55.788 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify (0 ms) 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-27T08:31:55.789 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (0 ms) 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP (0 ms total) 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:31:55.790 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-27T08:31:57.771 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-27T08:31:58.135 INFO:tasks.workunit.client.0.smithi097.stdout:notify 2024-03-27T08:31:58.149 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (378 ms) 2024-03-27T08:31:58.149 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-27T08:31:58.181 INFO:tasks.workunit.client.0.smithi097.stdout:notify 2024-03-27T08:31:58.194 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (46 ms) 2024-03-27T08:31:58.195 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-27T08:31:58.233 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (37 ms) 2024-03-27T08:31:58.233 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-27T08:31:58.264 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (31 ms) 2024-03-27T08:31:58.264 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-27T08:31:58.292 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22701040 notify_id 1705102016513 notifier_gid 15606 2024-03-27T08:31:58.305 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (42 ms) 2024-03-27T08:31:58.305 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-27T08:31:58.334 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22701040 notify_id 1705102016514 notifier_gid 15606 2024-03-27T08:31:58.348 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (43 ms) 2024-03-27T08:31:58.348 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-27T08:31:58.378 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22704912 notify_id 1705102016515 notifier_gid 15606 2024-03-27T08:31:58.393 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (44 ms) 2024-03-27T08:31:58.393 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-27T08:31:58.424 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22708144 notify_id 1705102016515 notifier_gid 15606 2024-03-27T08:31:58.438 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (45 ms) 2024-03-27T08:31:58.438 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-27T08:31:58.467 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22708144 notify_id 1705102016515 notifier_gid 15606 2024-03-27T08:31:58.482 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (44 ms) 2024-03-27T08:31:58.482 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-27T08:31:58.510 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22721408 notify_id 1705102016516 notifier_gid 15606 2024-03-27T08:31:58.524 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (42 ms) 2024-03-27T08:31:58.524 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-27T08:31:58.550 INFO:tasks.workunit.client.0.smithi097.stdout: trying... 2024-03-27T08:31:58.554 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22721408 notify_id 1705102016516 notifier_gid 15606 2024-03-27T08:31:59.553 INFO:tasks.workunit.client.0.smithi097.stdout: timed out 2024-03-27T08:31:59.564 INFO:tasks.workunit.client.0.smithi097.stdout: flushing 2024-03-27T08:32:01.555 INFO:tasks.workunit.client.0.smithi097.stdout: flushed 2024-03-27T08:32:01.555 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3030 ms) 2024-03-27T08:32:01.555 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-27T08:32:01.592 INFO:tasks.workunit.client.0.smithi097.stdout: trying... 2024-03-27T08:32:01.596 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22721408 notify_id 1705102016517 notifier_gid 15606 2024-03-27T08:32:02.594 INFO:tasks.workunit.client.0.smithi097.stdout: timed out 2024-03-27T08:32:02.609 INFO:tasks.workunit.client.0.smithi097.stdout: flushing 2024-03-27T08:32:04.597 INFO:tasks.workunit.client.0.smithi097.stdout: flushed 2024-03-27T08:32:04.598 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3042 ms) 2024-03-27T08:32:04.598 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-27T08:32:04.627 INFO:tasks.workunit.client.0.smithi097.stdout:List watches 2024-03-27T08:32:04.627 INFO:tasks.workunit.client.0.smithi097.stdout:notify2 2024-03-27T08:32:04.632 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22721408 notify_id 1705102016516 notifier_gid 15606 2024-03-27T08:32:07.637 INFO:tasks.workunit.client.0.smithi097.stdout:notify2 done 2024-03-27T08:32:07.638 INFO:tasks.workunit.client.0.smithi097.stdout:watch_check 2024-03-27T08:32:07.638 INFO:tasks.workunit.client.0.smithi097.stdout:unwatch2 2024-03-27T08:32:07.650 INFO:tasks.workunit.client.0.smithi097.stdout: flushing 2024-03-27T08:32:07.650 INFO:tasks.workunit.client.0.smithi097.stdout:done 2024-03-27T08:32:07.650 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3054 ms) 2024-03-27T08:32:07.650 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-27T08:32:07.681 INFO:tasks.workunit.client.0.smithi097.stdout:List watches 2024-03-27T08:32:07.681 INFO:tasks.workunit.client.0.smithi097.stdout:notify2 2024-03-27T08:32:07.685 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 22723552 notify_id 1709396983814 notifier_gid 15606 2024-03-27T08:32:10.691 INFO:tasks.workunit.client.0.smithi097.stdout:notify2 done 2024-03-27T08:32:10.691 INFO:tasks.workunit.client.0.smithi097.stdout:watch_check 2024-03-27T08:32:10.692 INFO:tasks.workunit.client.0.smithi097.stdout:unwatch2 2024-03-27T08:32:10.705 INFO:tasks.workunit.client.0.smithi097.stdout: flushing 2024-03-27T08:32:10.705 INFO:tasks.workunit.client.0.smithi097.stdout:done 2024-03-27T08:32:10.705 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3055 ms) 2024-03-27T08:32:10.733 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (12934 ms total) 2024-03-27T08:32:10.733 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:10.734 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:10.734 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 16 tests from 2 test suites ran. (14945 ms total) 2024-03-27T08:32:10.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 14 tests. 2024-03-27T08:32:10.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 2 tests, listed below: 2024-03-27T08:32:10.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-27T08:32:10.734 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-27T08:32:10.735 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:10.735 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:10.735 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_cmd 2024-03-27T08:32:10.756 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:32:10.756 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-27T08:32:10.756 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:10.756 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosCmd 2024-03-27T08:32:10.756 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.MonDescribe 2024-03-27T08:32:10.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.MonDescribe (35 ms) 2024-03-27T08:32:10.791 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.OSDCmd 2024-03-27T08:32:10.823 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.OSDCmd (33 ms) 2024-03-27T08:32:10.823 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.PGCmd 2024-03-27T08:32:13.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.PGCmd (2926 ms) 2024-03-27T08:32:13.750 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.WatchLog 2024-03-27T08:32:13.773 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:12.745175+0000 mon.a [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T08:32:13.774 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:12.745442+0000 mon.a [INF] from='client.? 172.21.15.97:0/3180847062' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-96758-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-27T08:32:14.752 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:13.782946+0000 client.admin [INF] onexx 2024-03-27T08:32:14.753 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:13.783060+0000 mon.a [INF] from='client.? 172.21.15.97:0/259986628' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-27T08:32:15.753 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:14.761424+0000 mon.a [INF] from='client.? 172.21.15.97:0/259986628' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-27T08:32:18.757 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:16.765449+0000 mon.a [INF] from='client.? 172.21.15.97:0/259986628' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-27T08:32:18.809 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:18.766349+0000 client.admin [INF] threexx 2024-03-27T08:32:18.809 INFO:tasks.workunit.client.0.smithi097.stdout:got: 2024-03-27T08:32:18.766823+0000 mon.a [INF] from='client.? 172.21.15.97:0/259986628' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-27T08:32:22.815 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.WatchLog (9065 ms) 2024-03-27T08:32:22.815 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosCmd (12059 ms total) 2024-03-27T08:32:22.815 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:22.815 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:22.815 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 4 tests from 1 test suite ran. (12059 ms total) 2024-03-27T08:32:22.816 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 4 tests. 2024-03-27T08:32:22.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:22.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:22.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_cmd_pp 2024-03-27T08:32:22.838 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:32:22.838 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-27T08:32:22.839 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:22.839 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosCmd 2024-03-27T08:32:22.839 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.MonDescribePP 2024-03-27T08:32:22.871 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.MonDescribePP (32 ms) 2024-03-27T08:32:22.871 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.OSDCmdPP 2024-03-27T08:32:22.903 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.OSDCmdPP (32 ms) 2024-03-27T08:32:22.903 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCmd.PGCmdPP 2024-03-27T08:32:25.832 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCmd.PGCmdPP (2928 ms) 2024-03-27T08:32:25.833 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from LibRadosCmd (2993 ms total) 2024-03-27T08:32:25.833 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:25.833 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:25.833 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 3 tests from 1 test suite ran. (2993 ms total) 2024-03-27T08:32:25.833 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 3 tests. 2024-03-27T08:32:25.834 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:25.834 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:25.834 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_service 2024-03-27T08:32:25.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:32:25.877+0000 7fcef5961840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:25.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:32:25.877+0000 7fcef5961840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:25.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:32:25.877+0000 7fcef5961840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:25.869 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-27T08:32:25.869 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:25.870 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosService 2024-03-27T08:32:25.870 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosService.RegisterEarly 2024-03-27T08:32:30.926 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosService.RegisterEarly (5057 ms) 2024-03-27T08:32:30.926 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosService.RegisterLate 2024-03-27T08:32:30.966 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosService.RegisterLate (41 ms) 2024-03-27T08:32:30.966 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosService.StatusFormat 2024-03-27T08:32:31.107 INFO:tasks.workunit.client.0.smithi097.stdout: cluster: 2024-03-27T08:32:31.107 INFO:tasks.workunit.client.0.smithi097.stdout: id: ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:32:31.107 INFO:tasks.workunit.client.0.smithi097.stdout: health: HEALTH_WARN 2024-03-27T08:32:31.107 INFO:tasks.workunit.client.0.smithi097.stdout: 1 pool(s) do not have an application enabled 2024-03-27T08:32:31.107 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: services: 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: mon: 1 daemons, quorum a (age 9m) 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: mgr: x(active, since 9m) 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: osd: 4 osds: 4 up (since 9m), 4 in (since 9m) 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: laundry: 1 daemon active (1 hosts) 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: data: 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: pools: 3 pools, 41 pgs 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: objects: 3 objects, 577 KiB 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: usage: 168 MiB used, 400 GiB / 400 GiB avail 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: pgs: 41 active+clean 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:31.108 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: cluster: 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: id: ab3c95de-2422-462d-839c-7edbd2518ef5 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: health: HEALTH_WARN 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: 1 pool(s) do not have an application enabled 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: services: 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: mon: 1 daemons, quorum a (age 9m) 2024-03-27T08:32:33.134 INFO:tasks.workunit.client.0.smithi097.stdout: mgr: x(active, since 9m) 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: osd: 4 osds: 4 up (since 9m), 4 in (since 9m) 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: foo: 16 portals active (1 hosts, 3 zones) 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: data: 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: pools: 3 pools, 41 pgs 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: objects: 3 objects, 577 KiB 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: usage: 168 MiB used, 400 GiB / 400 GiB avail 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: pgs: 41 active+clean 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:33.135 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:33.157 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosService.StatusFormat (2191 ms) 2024-03-27T08:32:33.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosService.Status 2024-03-27T08:32:53.197 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosService.Status (20039 ms) 2024-03-27T08:32:53.198 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosService (27328 ms total) 2024-03-27T08:32:53.198 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:32:53.198 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:32:53.198 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 4 tests from 1 test suite ran. (27328 ms total) 2024-03-27T08:32:53.198 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 4 tests. 2024-03-27T08:32:53.203 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:32:53.203 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:32:53.203 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_service_pp 2024-03-27T08:32:53.238 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:32:53.245+0000 7f4043660840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:53.238 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:32:53.245+0000 7f4043660840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:53.238 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:32:53.246+0000 7f4043660840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:32:53.238 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-27T08:32:53.239 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:32:53.239 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosServicePP 2024-03-27T08:32:53.239 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosServicePP.RegisterEarly 2024-03-27T08:32:58.277 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosServicePP.RegisterEarly (5039 ms) 2024-03-27T08:32:58.277 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosServicePP.RegisterLate 2024-03-27T08:32:58.331 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosServicePP.RegisterLate (54 ms) 2024-03-27T08:32:58.332 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosServicePP.Status 2024-03-27T08:33:18.371 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosServicePP.Status (20040 ms) 2024-03-27T08:33:18.372 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosServicePP.Close 2024-03-27T08:33:18.372 INFO:tasks.workunit.client.0.smithi097.stdout:attempt 0 of 20 2024-03-27T08:33:23.437 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosServicePP.Close (5066 ms) 2024-03-27T08:33:23.437 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 4 tests from LibRadosServicePP (30199 ms total) 2024-03-27T08:33:23.437 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:33:23.437 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:33:23.437 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 4 tests from 1 test suite ran. (30199 ms total) 2024-03-27T08:33:23.437 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 4 tests. 2024-03-27T08:33:23.439 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:33:23.440 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:33:23.440 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_c_write_operations 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 8 tests from 2 test suites. 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibradosCWriteOps 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibradosCWriteOps.NewDelete 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-27T08:33:23.461 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-27T08:33:23.462 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:33:23.462 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from LibRadosCWriteOps 2024-03-27T08:33:23.462 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.assertExists 2024-03-27T08:33:25.801 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.assertExists (2340 ms) 2024-03-27T08:33:25.801 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-27T08:33:28.811 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3010 ms) 2024-03-27T08:33:28.811 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.Xattrs 2024-03-27T08:33:31.821 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.Xattrs (3010 ms) 2024-03-27T08:33:31.821 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.Write 2024-03-27T08:33:34.844 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.Write (3023 ms) 2024-03-27T08:33:34.844 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.Exec 2024-03-27T08:33:37.852 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.Exec (3008 ms) 2024-03-27T08:33:37.852 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.WriteSame 2024-03-27T08:33:40.875 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.WriteSame (3023 ms) 2024-03-27T08:33:40.875 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] LibRadosCWriteOps.CmpExt 2024-03-27T08:33:50.758 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] LibRadosCWriteOps.CmpExt (9880 ms) 2024-03-27T08:33:50.758 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from LibRadosCWriteOps (27294 ms total) 2024-03-27T08:33:50.758 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:33:50.758 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:33:50.758 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 8 tests from 2 test suites ran. (27294 ms total) 2024-03-27T08:33:50.759 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 8 tests. 2024-03-27T08:33:50.759 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:33:50.759 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:33:50.759 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_c_read_operations 2024-03-27T08:33:50.784 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:33:50.785 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 17 tests from 1 test suite. 2024-03-27T08:33:50.785 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:33:50.785 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 17 tests from CReadOpsTest 2024-03-27T08:33:52.763 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.NewDelete 2024-03-27T08:33:52.763 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-27T08:33:52.763 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.SetOpFlags 2024-03-27T08:33:53.171 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.SetOpFlags (408 ms) 2024-03-27T08:33:53.171 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.AssertExists 2024-03-27T08:33:53.207 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.AssertExists (37 ms) 2024-03-27T08:33:53.208 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.AssertVersion 2024-03-27T08:33:53.255 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.AssertVersion (47 ms) 2024-03-27T08:33:53.255 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.CmpXattr 2024-03-27T08:33:53.404 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.CmpXattr (149 ms) 2024-03-27T08:33:53.404 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.Read 2024-03-27T08:33:53.447 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.Read (43 ms) 2024-03-27T08:33:53.447 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.Checksum 2024-03-27T08:33:53.487 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.Checksum (40 ms) 2024-03-27T08:33:53.487 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.RWOrderedRead 2024-03-27T08:33:53.513 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.RWOrderedRead (27 ms) 2024-03-27T08:33:53.513 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.ShortRead 2024-03-27T08:33:53.555 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.ShortRead (41 ms) 2024-03-27T08:33:53.555 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.Exec 2024-03-27T08:33:53.578 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.Exec (23 ms) 2024-03-27T08:33:53.578 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.ExecUserBuf 2024-03-27T08:33:53.610 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.ExecUserBuf (32 ms) 2024-03-27T08:33:53.610 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.Stat 2024-03-27T08:33:53.642 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.Stat (32 ms) 2024-03-27T08:33:53.642 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.Stat2 2024-03-27T08:33:53.678 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.Stat2 (36 ms) 2024-03-27T08:33:53.678 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.Omap 2024-03-27T08:33:53.762 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.Omap (84 ms) 2024-03-27T08:33:53.762 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.OmapNuls 2024-03-27T08:33:53.954 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.OmapNuls (192 ms) 2024-03-27T08:33:53.954 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.GetXattrs 2024-03-27T08:33:54.020 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.GetXattrs (66 ms) 2024-03-27T08:33:54.021 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] CReadOpsTest.CmpExt 2024-03-27T08:33:54.048 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] CReadOpsTest.CmpExt (28 ms) 2024-03-27T08:33:54.771 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 17 tests from CReadOpsTest (1285 ms total) 2024-03-27T08:33:54.771 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:33:54.771 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:33:54.771 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 17 tests from 1 test suite ran. (3986 ms total) 2024-03-27T08:33:54.771 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 17 tests. 2024-03-27T08:33:54.773 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:33:54.773 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:33:54.773 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-03-27T08:33:54.816 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:33:54.824+0000 7fe24cc356c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:33:54.816 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:33:54.824+0000 7fe24cc356c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:33:54.816 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:33:54.824+0000 7fe24cc356c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 1 test from 1 test suite. 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from ClsTestRemoteReads 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] ClsTestRemoteReads.TestGather 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2463-g022fc18a/rpm/el9/BUILD/ceph-19.0.0-2463-g022fc18a/src/test/librados/cls_remote_reads.cc:13: Skipped 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:Not supported by crimson yet. Skipped 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] ClsTestRemoteReads.TestGather (0 ms) 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from ClsTestRemoteReads (0 ms total) 2024-03-27T08:33:54.817 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:33:54.818 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:33:54.818 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 1 test from 1 test suite ran. (0 ms total) 2024-03-27T08:33:54.818 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 0 tests. 2024-03-27T08:33:54.818 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-27T08:33:54.818 INFO:tasks.workunit.client.0.smithi097.stdout:[ SKIPPED ] ClsTestRemoteReads.TestGather 2024-03-27T08:33:54.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:33:54.819 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:33:54.819 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_list_parallel 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: starting. 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: creating pool ceph_test_rados_list_parallel.smithi097-97441 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: created object 0... 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: created object 25... 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: created object 49... 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: finishing. 2024-03-27T08:33:57.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97442]: shutting down. 2024-03-27T08:33:57.845 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97443]: starting. 2024-03-27T08:33:57.845 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97443]: listing objects. 2024-03-27T08:33:57.846 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97443]: listed object 0... 2024-03-27T08:33:57.846 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97443]: listed object 25... 2024-03-27T08:33:57.846 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97443]: saw 50 objects 2024-03-27T08:33:57.846 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97443]: shutting down. 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: starting. 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: creating pool ceph_test_rados_list_parallel.smithi097-97441 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: created object 0... 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: created object 25... 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: created object 49... 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: finishing. 2024-03-27T08:33:58.716 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97478]: shutting down. 2024-03-27T08:33:58.989 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:33:58.989 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97479]: starting. 2024-03-27T08:33:58.989 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97479]: listing objects. 2024-03-27T08:33:58.989 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97479]: listed object 0... 2024-03-27T08:33:58.990 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97479]: listed object 25... 2024-03-27T08:33:58.990 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97479]: saw 43 objects 2024-03-27T08:33:58.990 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97479]: shutting down. 2024-03-27T08:33:59.242 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:33:59.243 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97480]: starting. 2024-03-27T08:33:59.243 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97480]: removed 25 objects... 2024-03-27T08:33:59.243 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97480]: removed half of the objects 2024-03-27T08:33:59.243 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97480]: removed 50 objects... 2024-03-27T08:33:59.243 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97480]: removed 50 objects 2024-03-27T08:33:59.243 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97480]: shutting down. 2024-03-27T08:33:59.825 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:33:59.825 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: starting. 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: creating pool ceph_test_rados_list_parallel.smithi097-97441 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: created object 0... 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: created object 25... 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: created object 49... 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: finishing. 2024-03-27T08:33:59.826 INFO:tasks.workunit.client.0.smithi097.stdout:process_6_[97529]: shutting down. 2024-03-27T08:34:00.119 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.119 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.119 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: starting. 2024-03-27T08:34:00.119 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: listing objects. 2024-03-27T08:34:00.119 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: listed object 0... 2024-03-27T08:34:00.120 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: listed object 25... 2024-03-27T08:34:00.120 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: listed object 50... 2024-03-27T08:34:00.120 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: saw 56 objects 2024-03-27T08:34:00.120 INFO:tasks.workunit.client.0.smithi097.stdout:process_7_[97530]: shutting down. 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:process_8_[97531]: starting. 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:process_8_[97531]: added 25 objects... 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:process_8_[97531]: added half of the objects 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:process_8_[97531]: added 50 objects... 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:process_8_[97531]: added 50 objects 2024-03-27T08:34:00.395 INFO:tasks.workunit.client.0.smithi097.stdout:process_8_[97531]: shutting down. 2024-03-27T08:34:00.974 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: starting. 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: creating pool ceph_test_rados_list_parallel.smithi097-97441 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: created object 0... 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: created object 25... 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: created object 49... 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: finishing. 2024-03-27T08:34:00.975 INFO:tasks.workunit.client.0.smithi097.stdout:process_9_[97580]: shutting down. 2024-03-27T08:34:01.329 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.329 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.329 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: starting. 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: listing objects. 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: listed object 0... 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: listed object 25... 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: listed object 50... 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: listed object 75... 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: saw 100 objects 2024-03-27T08:34:01.330 INFO:tasks.workunit.client.0.smithi097.stdout:process_10_[97581]: shutting down. 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:process_11_[97582]: starting. 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:process_11_[97582]: added 25 objects... 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:process_11_[97582]: added half of the objects 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:process_11_[97582]: added 50 objects... 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:process_11_[97582]: added 50 objects 2024-03-27T08:34:01.649 INFO:tasks.workunit.client.0.smithi097.stdout:process_11_[97582]: shutting down. 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:process_13_[97584]: starting. 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:process_13_[97584]: removed 25 objects... 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:process_13_[97584]: removed half of the objects 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:process_13_[97584]: removed 50 objects... 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:process_13_[97584]: removed 50 objects 2024-03-27T08:34:01.658 INFO:tasks.workunit.client.0.smithi097.stdout:process_13_[97584]: shutting down. 2024-03-27T08:34:01.690 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.690 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.690 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:01.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_12_[97583]: starting. 2024-03-27T08:34:01.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_12_[97583]: added 25 objects... 2024-03-27T08:34:01.690 INFO:tasks.workunit.client.0.smithi097.stdout:process_12_[97583]: added half of the objects 2024-03-27T08:34:01.691 INFO:tasks.workunit.client.0.smithi097.stdout:process_12_[97583]: added 50 objects... 2024-03-27T08:34:01.691 INFO:tasks.workunit.client.0.smithi097.stdout:process_12_[97583]: added 50 objects 2024-03-27T08:34:01.691 INFO:tasks.workunit.client.0.smithi097.stdout:process_12_[97583]: shutting down. 2024-03-27T08:34:02.266 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.266 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.266 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: starting. 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: creating pool ceph_test_rados_list_parallel.smithi097-97441 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: created object 0... 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: created object 25... 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: created object 49... 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: finishing. 2024-03-27T08:34:02.267 INFO:tasks.workunit.client.0.smithi097.stdout:process_14_[97665]: shutting down. 2024-03-27T08:34:02.555 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.555 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: starting. 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listing objects. 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listed object 0... 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listed object 25... 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listed object 50... 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listed object 75... 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listed object 100... 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: listed object 125... 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: saw 150 objects 2024-03-27T08:34:02.556 INFO:tasks.workunit.client.0.smithi097.stdout:process_15_[97666]: shutting down. 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:process_16_[97667]: starting. 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:process_16_[97667]: added 25 objects... 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:process_16_[97667]: added half of the objects 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:process_16_[97667]: added 50 objects... 2024-03-27T08:34:02.815 INFO:tasks.workunit.client.0.smithi097.stdout:process_16_[97667]: added 50 objects 2024-03-27T08:34:02.816 INFO:tasks.workunit.client.0.smithi097.stdout:process_16_[97667]: shutting down. 2024-03-27T08:34:03.784 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:03.784 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:03.784 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:03.784 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:03.784 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:03.785 INFO:tasks.workunit.client.0.smithi097.stdout:******* SUCCESS ********** 2024-03-27T08:34:03.785 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:34:03.785 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:03.786 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_open_pools_parallel 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: starting. 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: creating pool ceph_test_rados_open_pools_parallel.smithi097-97732 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: created object 0... 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: created object 25... 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: created object 49... 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: finishing. 2024-03-27T08:34:06.697 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97733]: shutting down. 2024-03-27T08:34:06.804 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97734]: starting. 2024-03-27T08:34:06.804 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97734]: rados_pool_create. 2024-03-27T08:34:06.804 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97734]: rados_ioctx_create. 2024-03-27T08:34:06.804 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97734]: shutting down. 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: starting. 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: creating pool ceph_test_rados_open_pools_parallel.smithi097-97732 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: created object 0... 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: created object 25... 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: created object 49... 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: finishing. 2024-03-27T08:34:09.714 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97769]: shutting down. 2024-03-27T08:34:09.825 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:09.825 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97770]: starting. 2024-03-27T08:34:09.825 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97770]: rados_pool_create. 2024-03-27T08:34:09.825 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97770]: rados_ioctx_create. 2024-03-27T08:34:09.825 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97770]: shutting down. 2024-03-27T08:34:09.826 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:09.826 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:09.826 INFO:tasks.workunit.client.0.smithi097.stdout:******* SUCCESS ********** 2024-03-27T08:34:09.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-27T08:34:09.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:09.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_delete_pools_parallel 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: starting. 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: creating pool ceph_test_rados_delete_pools_parallel.smithi097-97805 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: created object 0... 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: created object 25... 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: created object 49... 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: finishing. 2024-03-27T08:34:12.724 INFO:tasks.workunit.client.0.smithi097.stdout:process_1_[97806]: shutting down. 2024-03-27T08:34:12.837 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97807]: starting. 2024-03-27T08:34:12.838 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97807]: deleting pool ceph_test_rados_delete_pools_parallel.smithi097-97805 2024-03-27T08:34:12.838 INFO:tasks.workunit.client.0.smithi097.stdout:process_2_[97807]: shutting down. 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: starting. 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: creating pool ceph_test_rados_delete_pools_parallel.smithi097-97805 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: created object 0... 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: created object 25... 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: created object 49... 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: finishing. 2024-03-27T08:34:15.784 INFO:tasks.workunit.client.0.smithi097.stdout:process_3_[97842]: shutting down. 2024-03-27T08:34:15.977 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:15.978 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97844]: starting. 2024-03-27T08:34:15.978 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97844]: listing objects. 2024-03-27T08:34:15.978 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97844]: listed object 0... 2024-03-27T08:34:15.978 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97844]: listed object 25... 2024-03-27T08:34:15.978 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97844]: saw 50 objects 2024-03-27T08:34:15.978 INFO:tasks.workunit.client.0.smithi097.stdout:process_5_[97844]: shutting down. 2024-03-27T08:34:16.876 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:16.877 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97843]: starting. 2024-03-27T08:34:16.877 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97843]: deleting pool ceph_test_rados_delete_pools_parallel.smithi097-97805 2024-03-27T08:34:16.877 INFO:tasks.workunit.client.0.smithi097.stdout:process_4_[97843]: shutting down. 2024-03-27T08:34:16.879 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:16.879 INFO:tasks.workunit.client.0.smithi097.stdout:******************************* 2024-03-27T08:34:16.879 INFO:tasks.workunit.client.0.smithi097.stdout:******* SUCCESS ********** 2024-03-27T08:34:16.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:34:16.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:16.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:34:16.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' cls = ec_io ']' 2024-03-27T08:34:16.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' cls = ec_list ']' 2024-03-27T08:34:16.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_cls 2024-03-27T08:34:16.901 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:16.901 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 2 tests from 1 test suite. 2024-03-27T08:34:16.901 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:16.901 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from NeoRadosCls 2024-03-27T08:34:16.901 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosCls.DNE 2024-03-27T08:34:19.888 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosCls.DNE (2987 ms) 2024-03-27T08:34:19.888 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosCls.RemoteReads 2024-03-27T08:34:20.895 INFO:tasks.workunit.client.0.smithi097.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:34:21.896 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosCls.RemoteReads (2008 ms) 2024-03-27T08:34:21.896 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from NeoRadosCls (4995 ms total) 2024-03-27T08:34:21.897 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:34:21.897 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:34:21.897 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 2 tests from 1 test suite ran. (4995 ms total) 2024-03-27T08:34:21.897 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 2 tests. 2024-03-27T08:34:21.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:34:21.899 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:21.899 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:34:21.899 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' cmd = ec_io ']' 2024-03-27T08:34:21.899 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' cmd = ec_list ']' 2024-03-27T08:34:21.899 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_cmd 2024-03-27T08:34:21.921 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:21.921 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-27T08:34:21.921 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:21.921 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from NeoRadosCmd 2024-03-27T08:34:21.921 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosCmd.MonDescribe 2024-03-27T08:34:23.904 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosCmd.MonDescribe (1984 ms) 2024-03-27T08:34:23.905 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosCmd.OSDCmd 2024-03-27T08:34:25.913 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosCmd.OSDCmd (2008 ms) 2024-03-27T08:34:25.914 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosCmd.PGCmd 2024-03-27T08:34:28.929 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosCmd.PGCmd (3017 ms) 2024-03-27T08:34:28.930 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from NeoRadosCmd (7009 ms total) 2024-03-27T08:34:28.930 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:34:28.930 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:34:28.930 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 3 tests from 1 test suite ran. (7009 ms total) 2024-03-27T08:34:28.930 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 3 tests. 2024-03-27T08:34:28.932 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:34:28.932 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:28.932 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:34:28.932 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' handler_error = ec_io ']' 2024-03-27T08:34:28.932 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' handler_error = ec_list ']' 2024-03-27T08:34:28.932 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_handler_error 2024-03-27T08:34:28.949 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:28.949 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 1 test from 1 test suite. 2024-03-27T08:34:28.949 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:28.949 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from neocls_handler_error 2024-03-27T08:34:28.949 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] neocls_handler_error.test_handler_error 2024-03-27T08:34:31.941 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] neocls_handler_error.test_handler_error (2992 ms) 2024-03-27T08:34:31.942 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 1 test from neocls_handler_error (2993 ms total) 2024-03-27T08:34:31.942 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:34:31.942 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:34:31.942 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 1 test from 1 test suite ran. (2993 ms total) 2024-03-27T08:34:31.942 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 1 test. 2024-03-27T08:34:31.943 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:34:31.944 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:34:31.944 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:34:31.944 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' io = ec_io ']' 2024-03-27T08:34:31.944 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' io = ec_list ']' 2024-03-27T08:34:31.944 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_io 2024-03-27T08:34:31.961 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:34:31.961 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 14 tests from 1 test suite. 2024-03-27T08:34:31.961 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:34:31.961 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from NeoRadosIo 2024-03-27T08:34:31.961 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.Limits 2024-03-27T08:34:32.760 INFO:tasks.workunit.client.0.smithi097.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:34:33.767 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.Limits (1807 ms) 2024-03-27T08:34:33.768 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.SimpleWrite 2024-03-27T08:34:36.778 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.SimpleWrite (3010 ms) 2024-03-27T08:34:36.778 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.ReadOp 2024-03-27T08:34:39.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.ReadOp (3009 ms) 2024-03-27T08:34:39.787 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.SparseRead 2024-03-27T08:34:42.800 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.SparseRead (3013 ms) 2024-03-27T08:34:42.800 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.RoundTrip 2024-03-27T08:34:45.816 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.RoundTrip (3016 ms) 2024-03-27T08:34:45.816 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-27T08:34:48.825 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.ReadIntoBuufferlist (3010 ms) 2024-03-27T08:34:48.826 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-27T08:34:51.834 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3009 ms) 2024-03-27T08:34:51.835 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-27T08:34:54.852 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.WriteFullRoundTrip (3017 ms) 2024-03-27T08:34:54.852 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-27T08:34:57.766 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.AppendRoundTrip (2915 ms) 2024-03-27T08:34:57.767 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.Trunc 2024-03-27T08:35:00.872 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.Trunc (3105 ms) 2024-03-27T08:35:00.872 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.Remove 2024-03-27T08:35:03.886 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.Remove (3014 ms) 2024-03-27T08:35:03.886 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-27T08:35:06.901 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.XattrsRoundTrip (3015 ms) 2024-03-27T08:35:06.901 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.RmXattr 2024-03-27T08:35:09.906 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.RmXattr (3006 ms) 2024-03-27T08:35:09.906 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosIo.GetXattrs 2024-03-27T08:35:12.925 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosIo.GetXattrs (3018 ms) 2024-03-27T08:35:12.925 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from NeoRadosIo (40964 ms total) 2024-03-27T08:35:12.925 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:35:12.926 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:12.926 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 14 tests from 1 test suite ran. (40964 ms total) 2024-03-27T08:35:12.926 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 14 tests. 2024-03-27T08:35:12.929 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:35:12.929 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:12.929 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:35:12.929 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' ec_io = ec_io ']' 2024-03-27T08:35:12.929 INFO:tasks.workunit.client.0.smithi097.stdout:Skipping EC with Crimson 2024-03-27T08:35:12.930 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'Skipping EC with Crimson' 2024-03-27T08:35:12.930 INFO:tasks.workunit.client.0.smithi097.stderr:+ continue 2024-03-27T08:35:12.930 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:35:12.930 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:12.930 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:35:12.930 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' list = ec_io ']' 2024-03-27T08:35:12.931 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' list = ec_list ']' 2024-03-27T08:35:12.931 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_list 2024-03-27T08:35:12.953 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:35:12.953 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-27T08:35:12.953 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:12.953 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from NeoradosList 2024-03-27T08:35:12.953 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoradosList.ListObjects 2024-03-27T08:35:15.935 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoradosList.ListObjects (2982 ms) 2024-03-27T08:35:15.935 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoradosList.ListObjectsNS 2024-03-27T08:35:18.945 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoradosList.ListObjectsNS (3010 ms) 2024-03-27T08:35:18.945 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoradosList.ListObjectsMany 2024-03-27T08:35:28.140 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoradosList.ListObjectsMany (9194 ms) 2024-03-27T08:35:28.140 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 3 tests from NeoradosList (15186 ms total) 2024-03-27T08:35:28.140 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:35:28.140 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:28.140 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 3 tests from 1 test suite ran. (15186 ms total) 2024-03-27T08:35:28.140 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 3 tests. 2024-03-27T08:35:28.141 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:35:28.142 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:28.142 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:35:28.142 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' ec_list = ec_io ']' 2024-03-27T08:35:28.142 INFO:tasks.workunit.client.0.smithi097.stdout:Skipping EC with Crimson 2024-03-27T08:35:28.142 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' ec_list = ec_list ']' 2024-03-27T08:35:28.142 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'Skipping EC with Crimson' 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ continue 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' misc = ec_io ']' 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' misc = ec_list ']' 2024-03-27T08:35:28.143 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_misc 2024-03-27T08:35:28.161 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:35:28.161 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 12 tests from 1 test suite. 2024-03-27T08:35:28.161 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:28.161 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 12 tests from NeoRadosMisc 2024-03-27T08:35:28.162 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.Version 2024-03-27T08:35:30.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.Version (1986 ms) 2024-03-27T08:35:30.148 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-27T08:35:32.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.WaitOSDMap (2010 ms) 2024-03-27T08:35:32.158 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.LongName 2024-03-27T08:35:34.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.LongName (2621 ms) 2024-03-27T08:35:34.779 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.LongLocator 2024-03-27T08:35:37.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.LongLocator (3013 ms) 2024-03-27T08:35:37.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.LongNamespace 2024-03-27T08:35:40.808 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.LongNamespace (3016 ms) 2024-03-27T08:35:40.809 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.LongAttrName 2024-03-27T08:35:43.820 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.LongAttrName (3012 ms) 2024-03-27T08:35:43.820 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.Exec 2024-03-27T08:35:46.834 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.Exec (3014 ms) 2024-03-27T08:35:46.834 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.Operate1 2024-03-27T08:35:49.843 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.Operate1 (3010 ms) 2024-03-27T08:35:49.844 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.Operate2 2024-03-27T08:35:52.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.Operate2 (2936 ms) 2024-03-27T08:35:52.780 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.BigObject 2024-03-27T08:35:55.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.BigObject (3006 ms) 2024-03-27T08:35:55.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.BigAttr 2024-03-27T08:35:57.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.BigAttr (2006 ms) 2024-03-27T08:35:57.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosMisc.WriteSame 2024-03-27T08:35:58.800 INFO:tasks.workunit.client.0.smithi097.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:35:59.803 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosMisc.WriteSame (2010 ms) 2024-03-27T08:35:59.803 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 12 tests from NeoRadosMisc (31641 ms total) 2024-03-27T08:35:59.803 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:35:59.803 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:35:59.803 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 12 tests from 1 test suite ran. (31641 ms total) 2024-03-27T08:35:59.803 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 12 tests. 2024-03-27T08:35:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:35:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:35:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:35:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' pool = ec_io ']' 2024-03-27T08:35:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' pool = ec_list ']' 2024-03-27T08:35:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_pool 2024-03-27T08:35:59.831 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:35:59.831 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 6 tests from 1 test suite. 2024-03-27T08:35:59.831 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:35:59.831 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 6 tests from NeoRadosPools 2024-03-27T08:35:59.831 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosPools.PoolList 2024-03-27T08:36:01.816 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosPools.PoolList (1984 ms) 2024-03-27T08:36:01.816 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosPools.PoolLookup 2024-03-27T08:36:03.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosPools.PoolLookup (1969 ms) 2024-03-27T08:36:03.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-27T08:36:05.835 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosPools.PoolLookupOtherInstance (2050 ms) 2024-03-27T08:36:05.835 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosPools.PoolDelete 2024-03-27T08:36:09.848 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosPools.PoolDelete (4013 ms) 2024-03-27T08:36:09.848 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-27T08:36:11.856 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosPools.PoolCreateDelete (2008 ms) 2024-03-27T08:36:11.856 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-27T08:36:13.784 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosPools.PoolCreateWithCrushRule (1928 ms) 2024-03-27T08:36:13.784 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 6 tests from NeoRadosPools (13952 ms total) 2024-03-27T08:36:13.785 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:36:13.785 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:36:13.785 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 6 tests from 1 test suite ran. (13952 ms total) 2024-03-27T08:36:13.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 6 tests. 2024-03-27T08:36:13.787 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:36:13.787 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:36:13.788 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:36:13.788 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' read_operations = ec_io ']' 2024-03-27T08:36:13.788 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' read_operations = ec_list ']' 2024-03-27T08:36:13.788 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_read_operations 2024-03-27T08:36:13.811 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:36:13.811 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 14 tests from 1 test suite. 2024-03-27T08:36:13.811 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:36:13.811 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from NeoRadosReadOps 2024-03-27T08:36:13.811 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-27T08:36:16.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.SetOpFlags (2985 ms) 2024-03-27T08:36:16.796 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.AssertExists 2024-03-27T08:36:19.811 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.AssertExists (3015 ms) 2024-03-27T08:36:19.811 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.AssertVersion 2024-03-27T08:36:22.785 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.AssertVersion (2975 ms) 2024-03-27T08:36:22.786 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.CmpXattr 2024-03-27T08:36:25.837 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.CmpXattr (3051 ms) 2024-03-27T08:36:25.837 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.Read 2024-03-27T08:36:28.853 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.Read (3017 ms) 2024-03-27T08:36:28.854 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.Checksum 2024-03-27T08:36:31.865 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.Checksum (3011 ms) 2024-03-27T08:36:31.865 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-27T08:36:34.877 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.RWOrderedRead (3012 ms) 2024-03-27T08:36:34.878 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.ShortRead 2024-03-27T08:36:37.891 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.ShortRead (3014 ms) 2024-03-27T08:36:37.891 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.Exec 2024-03-27T08:36:40.921 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.Exec (3030 ms) 2024-03-27T08:36:40.921 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.Stat 2024-03-27T08:36:43.932 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.Stat (3012 ms) 2024-03-27T08:36:43.933 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.Omap 2024-03-27T08:36:46.944 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.Omap (3011 ms) 2024-03-27T08:36:46.944 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.OmapNuls 2024-03-27T08:36:49.953 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.OmapNuls (3009 ms) 2024-03-27T08:36:49.953 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.GetXattrs 2024-03-27T08:36:52.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.GetXattrs (2839 ms) 2024-03-27T08:36:52.792 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosReadOps.CmpExt 2024-03-27T08:36:55.797 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosReadOps.CmpExt (3006 ms) 2024-03-27T08:36:55.797 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 14 tests from NeoRadosReadOps (41987 ms total) 2024-03-27T08:36:55.798 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:36:55.798 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:36:55.798 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 14 tests from 1 test suite ran. (41987 ms total) 2024-03-27T08:36:55.798 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 14 tests. 2024-03-27T08:36:55.801 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:36:55.801 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:36:55.801 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:36:55.801 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' snapshots = ec_io ']' 2024-03-27T08:36:55.801 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' snapshots = ec_list ']' 2024-03-27T08:36:55.801 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_snapshots 2024-03-27T08:36:55.817 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:36:55.818 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 11 tests from 2 test suites. 2024-03-27T08:36:55.818 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:36:55.818 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from NeoRadosSnapshots 2024-03-27T08:36:55.818 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSnapshots.SnapList 2024-03-27T08:37:00.817 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSnapshots.SnapList (4999 ms) 2024-03-27T08:37:00.817 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-27T08:37:05.841 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSnapshots.SnapRemove (5024 ms) 2024-03-27T08:37:05.841 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSnapshots.Rollback 2024-03-27T08:37:09.858 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSnapshots.Rollback (4016 ms) 2024-03-27T08:37:09.858 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-27T08:37:14.799 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSnapshots.SnapGetName (4941 ms) 2024-03-27T08:37:14.799 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-27T08:37:21.835 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSnapshots.SnapCreateRemove (7037 ms) 2024-03-27T08:37:21.836 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 5 tests from NeoRadosSnapshots (26018 ms total) 2024-03-27T08:37:21.836 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:37:21.836 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-27T08:37:21.836 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-27T08:37:26.855 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSelfManagedSnaps.Snap (5019 ms) 2024-03-27T08:37:26.855 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-27T08:37:27.865 INFO:tasks.workunit.client.0.smithi097.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:37:28.867 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSelfManagedSnaps.Rollback (2012 ms) 2024-03-27T08:37:28.867 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-27T08:37:29.876 INFO:tasks.workunit.client.0.smithi097.stderr:Not supported by crimson yet. Skipped 2024-03-27T08:37:30.878 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (2011 ms) 2024-03-27T08:37:30.878 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-27T08:37:36.907 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6029 ms) 2024-03-27T08:37:36.907 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-27T08:37:40.930 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4021 ms) 2024-03-27T08:37:40.930 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-27T08:37:43.933 INFO:tasks.workunit.client.0.smithi097.stdout:Deleting snap 3 in pool ReusePurgedSnapsmithi097-98752-11. 2024-03-27T08:37:44.938 INFO:tasks.workunit.client.0.smithi097.stdout:Waiting for snaps to purge. 2024-03-27T08:38:00.817 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19890 ms) 2024-03-27T08:38:00.818 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps (38982 ms total) 2024-03-27T08:38:00.818 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:38:00.818 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:38:00.818 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 11 tests from 2 test suites ran. (65000 ms total) 2024-03-27T08:38:00.818 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 11 tests. 2024-03-27T08:38:00.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:38:00.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:00.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:38:00.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' watch_notify = ec_io ']' 2024-03-27T08:38:00.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' watch_notify = ec_list ']' 2024-03-27T08:38:00.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_watch_notify 2024-03-27T08:38:00.842 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:38:00.842 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 2 tests from 1 test suite. 2024-03-27T08:38:00.843 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:38:00.843 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from NeoRadosWatchNotify 2024-03-27T08:38:00.843 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-27T08:38:03.277 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 12313680 notify_id 2808908611588 notifier_gid 16004 2024-03-27T08:38:03.832 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWatchNotify.WatchNotify (2990 ms) 2024-03-27T08:38:03.832 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-27T08:38:06.280 INFO:tasks.workunit.client.0.smithi097.stdout:Trying... 2024-03-27T08:38:06.292 INFO:tasks.workunit.client.0.smithi097.stdout:handle_notify cookie 25478704 notify_id 2821793513477 notifier_gid 16007 2024-03-27T08:38:06.292 INFO:tasks.workunit.client.0.smithi097.stdout:Waiting for 3.000000000s 2024-03-27T08:38:07.328 INFO:tasks.workunit.client.0.smithi097.stdout:Timed out. 2024-03-27T08:38:07.348 INFO:tasks.workunit.client.0.smithi097.stdout:Flushing... 2024-03-27T08:38:07.349 INFO:tasks.workunit.client.0.smithi097.stdout:Flushed... 2024-03-27T08:38:10.855 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7022 ms) 2024-03-27T08:38:10.855 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 2 tests from NeoRadosWatchNotify (10012 ms total) 2024-03-27T08:38:10.855 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:38:10.855 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:38:10.855 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 2 tests from 1 test suite ran. (10012 ms total) 2024-03-27T08:38:10.856 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 2 tests. 2024-03-27T08:38:10.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-27T08:38:10.857 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:10.857 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-27T08:38:10.857 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' write_operations = ec_io ']' 2024-03-27T08:38:10.857 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' write_operations = ec_list ']' 2024-03-27T08:38:10.857 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_write_operations 2024-03-27T08:38:10.876 INFO:tasks.workunit.client.0.smithi097.stdout:Running main() from gmock_main.cc 2024-03-27T08:38:10.876 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] Running 7 tests from 1 test suite. 2024-03-27T08:38:10.876 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment set-up. 2024-03-27T08:38:10.876 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from NeoRadosWriteOps 2024-03-27T08:38:10.877 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.AssertExists 2024-03-27T08:38:13.909 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.AssertExists (3032 ms) 2024-03-27T08:38:13.909 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-27T08:38:16.922 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.AssertVersion (3013 ms) 2024-03-27T08:38:16.922 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.Xattrs 2024-03-27T08:38:19.937 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.Xattrs (3015 ms) 2024-03-27T08:38:19.937 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.Write 2024-03-27T08:38:22.950 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.Write (3013 ms) 2024-03-27T08:38:22.951 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.Exec 2024-03-27T08:38:25.966 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.Exec (3015 ms) 2024-03-27T08:38:25.966 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.WriteSame 2024-03-27T08:38:28.975 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.WriteSame (3009 ms) 2024-03-27T08:38:28.975 INFO:tasks.workunit.client.0.smithi097.stdout:[ RUN ] NeoRadosWriteOps.CmpExt 2024-03-27T08:38:39.182 INFO:tasks.workunit.client.0.smithi097.stdout:[ OK ] NeoRadosWriteOps.CmpExt (10208 ms) 2024-03-27T08:38:39.183 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] 7 tests from NeoRadosWriteOps (28306 ms total) 2024-03-27T08:38:39.183 INFO:tasks.workunit.client.0.smithi097.stdout: 2024-03-27T08:38:39.183 INFO:tasks.workunit.client.0.smithi097.stdout:[----------] Global test environment tear-down 2024-03-27T08:38:39.183 INFO:tasks.workunit.client.0.smithi097.stdout:[==========] 7 tests from 1 test suite ran. (28306 ms total) 2024-03-27T08:38:39.183 INFO:tasks.workunit.client.0.smithi097.stdout:[ PASSED ] 7 tests. 2024-03-27T08:38:39.185 INFO:tasks.workunit.client.0.smithi097.stderr:+ ret=0 2024-03-27T08:38:39.185 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -eq 1 ']' 2024-03-27T08:38:39.185 INFO:tasks.workunit.client.0.smithi097.stderr:+ exit 0 2024-03-27T08:38:39.185 INFO:tasks.workunit.client.0.smithi097.stderr:+ cleanup 2024-03-27T08:38:39.185 INFO:tasks.workunit.client.0.smithi097.stderr:+ pkill -P 33175 2024-03-27T08:38:39.274 INFO:tasks.workunit.client.0.smithi097.stderr:+ true 2024-03-27T08:38:39.274 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-27T08:38:39.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-27T08:38:39.344 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-27T08:38:39.346 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-27T08:38:39.346 DEBUG:teuthology.orchestra.run.smithi097:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=eae914f097640d21427554e7f6aa6f1e9c4a7748 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-27T08:38:39.405 INFO:tasks.workunit.client.0.smithi097.stderr:++ uuidgen 2024-03-27T08:38:39.418 INFO:tasks.workunit.client.0.smithi097.stderr:+ p=97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:38:39.418 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool create 97a8ed02-de19-41fe-985b-30a778c6d918 12 2024-03-27T08:38:39.577 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.574+0000 7f6cd50ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:39.578 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.587+0000 7f6cd50ed640 1 Processor -- start 2024-03-27T08:38:39.579 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.587+0000 7f6cd50ed640 1 -- start start 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.588+0000 7f6cd50ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cd006d3d0 0x7f6cd01318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cd50ed640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd00a0f50 con 0x7f6cd0131de0 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cd50ed640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd006e270 con 0x7f6cd006d3d0 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cced9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cd006d3d0 0x7f6cd01318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cced9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cd006d3d0 0x7f6cd01318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50828/0 (socket says 172.21.15.97:50828) 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cced9d640 1 -- 172.21.15.97:0/2434410334 learned_addr learned my addr 172.21.15.97:0/2434410334 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cce59c640 1 --1- 172.21.15.97:0/2434410334 >> v1:172.21.15.97:6789/0 conn(0x7f6cd0131de0 0x7f6cd009fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35364/0 (socket says 172.21.15.97:35364) 2024-03-27T08:38:39.581 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6ccdd9b640 1 -- 172.21.15.97:0/2434410334 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913363944 0 0) 0x7f6cd00a0f50 con 0x7f6cd0131de0 2024-03-27T08:38:39.582 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cced9d640 1 -- 172.21.15.97:0/2434410334 >> v1:172.21.15.97:6789/0 conn(0x7f6cd0131de0 legacy=0x7f6cd009fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:39.582 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.589+0000 7f6cced9d640 1 -- 172.21.15.97:0/2434410334 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd006df60 con 0x7f6cd006d3d0 2024-03-27T08:38:39.582 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.590+0000 7f6cced9d640 1 --2- 172.21.15.97:0/2434410334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd01318a0 crc :-1 s=READY pgs=12859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266a313571e144ab server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:39.582 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.590+0000 7f6ccdd9b640 1 -- 172.21.15.97:0/2434410334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc0009e50 con 0x7f6cd006d3d0 2024-03-27T08:38:39.582 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.590+0000 7f6ccdd9b640 1 -- 172.21.15.97:0/2434410334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6cc000e230 con 0x7f6cd006d3d0 2024-03-27T08:38:39.582 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.590+0000 7f6ccdd9b640 1 -- 172.21.15.97:0/2434410334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc000e4d0 con 0x7f6cd006d3d0 2024-03-27T08:38:39.583 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.591+0000 7f6cd50ed640 1 -- 172.21.15.97:0/2434410334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 msgr2=0x7f6cd01318a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:39.583 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.592+0000 7f6cd50ed640 1 --2- 172.21.15.97:0/2434410334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd01318a0 crc :-1 s=READY pgs=12859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:39.583 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.592+0000 7f6cd50ed640 1 -- 172.21.15.97:0/2434410334 shutdown_connections 2024-03-27T08:38:39.583 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.592+0000 7f6cd50ed640 1 --2- 172.21.15.97:0/2434410334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd01318a0 unknown :-1 s=CLOSED pgs=12859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:39.584 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.592+0000 7f6cd50ed640 1 -- 172.21.15.97:0/2434410334 >> 172.21.15.97:0/2434410334 conn(0x7f6cd00a5be0 msgr2=0x7f6cd00a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:39.584 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.592+0000 7f6cd50ed640 1 -- 172.21.15.97:0/2434410334 shutdown_connections 2024-03-27T08:38:39.584 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.592+0000 7f6cd50ed640 1 -- 172.21.15.97:0/2434410334 wait complete. 2024-03-27T08:38:39.584 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.593+0000 7f6cd50ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:39.587 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.596+0000 7f6cd50ed640 1 Processor -- start 2024-03-27T08:38:39.588 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.596+0000 7f6cd50ed640 1 -- start start 2024-03-27T08:38:39.589 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.598+0000 7f6cd50ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd011fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:39.589 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.598+0000 7f6cd50ed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0124020 con 0x7f6cd006d3d0 2024-03-27T08:38:39.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.598+0000 7f6cced9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd011fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:39.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.598+0000 7f6cced9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd011fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50840/0 (socket says 172.21.15.97:50840) 2024-03-27T08:38:39.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.598+0000 7f6cced9d640 1 -- 172.21.15.97:0/1381788705 learned_addr learned my addr 172.21.15.97:0/1381788705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:39.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.598+0000 7f6cced9d640 1 -- 172.21.15.97:0/1381788705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd01212e0 con 0x7f6cd006d3d0 2024-03-27T08:38:39.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.599+0000 7f6cced9d640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd011fe20 crc :-1 s=READY pgs=12860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:39.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.599+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc000ee50 con 0x7f6cd006d3d0 2024-03-27T08:38:39.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.599+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6cc00145c0 con 0x7f6cd006d3d0 2024-03-27T08:38:39.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.599+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd0120f40 con 0x7f6cd006d3d0 2024-03-27T08:38:39.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.599+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6cc0014860 con 0x7f6cd006d3d0 2024-03-27T08:38:39.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.599+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01229c0 con 0x7f6cd006d3d0 2024-03-27T08:38:39.592 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.600+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f6cc00149c0 con 0x7f6cd006d3d0 2024-03-27T08:38:39.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.601+0000 7f6cb77fe640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6cbc038840 0x7f6cbc03ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:39.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.601+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(683..683 src has 399..683) ==== 3403+0+0 (crc 0 0 0) 0x7f6cc0049930 con 0x7f6cd006d3d0 2024-03-27T08:38:39.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.601+0000 7f6cce59c640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6cbc038840 0x7f6cbc03ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:39.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.601+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd0074a60 con 0x7f6cd006d3d0 2024-03-27T08:38:39.594 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.602+0000 7f6cce59c640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6cbc038840 0x7f6cbc03ad50 crc :-1 s=READY pgs=3827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:39.603 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.611+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f6cc0012c00 con 0x7f6cd006d3d0 2024-03-27T08:38:39.819 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:39.827+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pg_num": 12} v 0) -- 0x7f6cd01212c0 con 0x7f6cd006d3d0 2024-03-27T08:38:40.186 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.194+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pg_num": 12}]=0 pool '97a8ed02-de19-41fe-985b-30a778c6d918' created v684) ==== 176+0+0 (crc 0 0 0) 0x7f6cc00127f0 con 0x7f6cd006d3d0 2024-03-27T08:38:40.385 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.393+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pg_num": 12} v 0) -- 0x7f6cd0122c10 con 0x7f6cd006d3d0 2024-03-27T08:38:40.385 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.394+0000 7f6cb77fe640 1 -- 172.21.15.97:0/1381788705 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pg_num": 12}]=0 pool '97a8ed02-de19-41fe-985b-30a778c6d918' already exists v684) ==== 183+0+0 (crc 0 0 0) 0x7f6cc00129d0 con 0x7f6cd006d3d0 2024-03-27T08:38:40.385 INFO:tasks.workunit.client.0.smithi097.stderr:pool '97a8ed02-de19-41fe-985b-30a778c6d918' already exists 2024-03-27T08:38:40.389 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.397+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6cbc038840 msgr2=0x7f6cbc03ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.397+0000 7f6cd50ed640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6cbc038840 0x7f6cbc03ad50 crc :-1 s=READY pgs=3827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.397+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 msgr2=0x7f6cd011fe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.397+0000 7f6cd50ed640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd011fe20 crc :-1 s=READY pgs=12860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.398+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 shutdown_connections 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.398+0000 7f6cd50ed640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6cbc038840 0x7f6cbc03ad50 unknown :-1 s=CLOSED pgs=3827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.398+0000 7f6cd50ed640 1 --2- 172.21.15.97:0/1381788705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd006d3d0 0x7f6cd011fe20 unknown :-1 s=CLOSED pgs=12860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.398+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 >> 172.21.15.97:0/1381788705 conn(0x7f6cd00a5be0 msgr2=0x7f6cd00a20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.398+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 shutdown_connections 2024-03-27T08:38:40.390 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.398+0000 7f6cd50ed640 1 -- 172.21.15.97:0/1381788705 wait complete. 2024-03-27T08:38:40.408 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_objects 10 2024-03-27T08:38:40.528 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.528+0000 7ff2be79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:40.529 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.538+0000 7ff2be79d640 1 Processor -- start 2024-03-27T08:38:40.529 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.538+0000 7ff2be79d640 1 -- start start 2024-03-27T08:38:40.531 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2be79d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2b8137b10 0x7ff2b8139f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:40.531 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2be79d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b806eb50 con 0x7ff2b809f6e0 2024-03-27T08:38:40.531 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2be79d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b809f2f0 con 0x7ff2b8137b10 2024-03-27T08:38:40.531 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2b8137b10 0x7ff2b8139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:40.531 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2b8137b10 0x7ff2b8139f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50844/0 (socket says 172.21.15.97:50844) 2024-03-27T08:38:40.531 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff2b809f6e0 0x7ff2b809faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35370/0 (socket says 172.21.15.97:35370) 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b77fe640 1 -- 172.21.15.97:0/3274823873 learned_addr learned my addr 172.21.15.97:0/3274823873 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b77fe640 1 -- 172.21.15.97:0/3274823873 >> v1:172.21.15.97:6789/0 conn(0x7ff2b809f6e0 legacy=0x7ff2b809faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b6ffd640 1 -- 172.21.15.97:0/3274823873 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974657440 0 0) 0x7ff2b806eb50 con 0x7ff2b809f6e0 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b77fe640 1 -- 172.21.15.97:0/3274823873 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b80a0200 con 0x7ff2b8137b10 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b77fe640 1 --2- 172.21.15.97:0/3274823873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b8137b10 0x7ff2b8139f30 crc :-1 s=READY pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89bc315b1ddc2fa server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.539+0000 7ff2b6ffd640 1 -- 172.21.15.97:0/3274823873 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff2ac009e50 con 0x7ff2b8137b10 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2b6ffd640 1 -- 172.21.15.97:0/3274823873 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff2ac00e230 con 0x7ff2b8137b10 2024-03-27T08:38:40.532 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2b6ffd640 1 -- 172.21.15.97:0/3274823873 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff2ac00e4d0 con 0x7ff2b8137b10 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 -- 172.21.15.97:0/3274823873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b8137b10 msgr2=0x7ff2b8139f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 --2- 172.21.15.97:0/3274823873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b8137b10 0x7ff2b8139f30 crc :-1 s=READY pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 -- 172.21.15.97:0/3274823873 shutdown_connections 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 --2- 172.21.15.97:0/3274823873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b8137b10 0x7ff2b8139f30 unknown :-1 s=CLOSED pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 -- 172.21.15.97:0/3274823873 >> 172.21.15.97:0/3274823873 conn(0x7ff2b80a1860 msgr2=0x7ff2b80a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 -- 172.21.15.97:0/3274823873 shutdown_connections 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.540+0000 7ff2be79d640 1 -- 172.21.15.97:0/3274823873 wait complete. 2024-03-27T08:38:40.533 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.541+0000 7ff2be79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.542+0000 7ff2be79d640 1 Processor -- start 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.542+0000 7ff2be79d640 1 -- start start 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.543+0000 7ff2be79d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 0x7ff2b811f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.543+0000 7ff2be79d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8120780 con 0x7ff2b809f6e0 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.543+0000 7ff2b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 0x7ff2b811f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.543+0000 7ff2b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 0x7ff2b811f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50846/0 (socket says 172.21.15.97:50846) 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.543+0000 7ff2b7fff640 1 -- 172.21.15.97:0/3647633178 learned_addr learned my addr 172.21.15.97:0/3647633178 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:40.536 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.543+0000 7ff2b7fff640 1 -- 172.21.15.97:0/3647633178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b811f150 con 0x7ff2b809f6e0 2024-03-27T08:38:40.537 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.544+0000 7ff2b7fff640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 0x7ff2b811f5b0 crc :-1 s=READY pgs=12863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:40.537 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.544+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff2a8004b80 con 0x7ff2b809f6e0 2024-03-27T08:38:40.537 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.544+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff2a8004ce0 con 0x7ff2b809f6e0 2024-03-27T08:38:40.537 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.544+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b8121ea0 con 0x7ff2b809f6e0 2024-03-27T08:38:40.537 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.544+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff2a8007120 con 0x7ff2b809f6e0 2024-03-27T08:38:40.537 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.544+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b8122780 con 0x7ff2b809f6e0 2024-03-27T08:38:40.538 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.545+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff2a80072d0 con 0x7ff2b809f6e0 2024-03-27T08:38:40.538 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.545+0000 7ff2b4ff9640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff2a40387a0 0x7ff2a403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:40.538 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.545+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(684..684 src has 399..684) ==== 3778+0+0 (crc 0 0 0) 0x7ff2a8048590 con 0x7ff2b809f6e0 2024-03-27T08:38:40.538 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.545+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b80693c0 con 0x7ff2b809f6e0 2024-03-27T08:38:40.538 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.545+0000 7ff2b77fe640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff2a40387a0 0x7ff2a403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:40.538 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.546+0000 7ff2b77fe640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff2a40387a0 0x7ff2a403acb0 crc :-1 s=READY pgs=3828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:40.542 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.551+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7ff2a800f230 con 0x7ff2b809f6e0 2024-03-27T08:38:40.725 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:40.734+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "10"} v 0) -- 0x7ff2b8075190 con 0x7ff2b809f6e0 2024-03-27T08:38:41.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:41.201+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v685) ==== 223+0+0 (crc 0 0 0) 0x7ff2a800a5d0 con 0x7ff2b809f6e0 2024-03-27T08:38:41.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:41.356+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "10"} v 0) -- 0x7ff2b8120f40 con 0x7ff2b809f6e0 2024-03-27T08:38:42.196 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.204+0000 7ff2b4ff9640 1 -- 172.21.15.97:0/3647633178 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v686) ==== 223+0+0 (crc 0 0 0) 0x7ff2a800a7b0 con 0x7ff2b809f6e0 2024-03-27T08:38:42.196 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 10 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff2a40387a0 msgr2=0x7ff2a403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff2a40387a0 0x7ff2a403acb0 crc :-1 s=READY pgs=3828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 msgr2=0x7ff2b811f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 0x7ff2b811f5b0 crc :-1 s=READY pgs=12863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 shutdown_connections 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff2a40387a0 0x7ff2a403acb0 unknown :-1 s=CLOSED pgs=3828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 --2- 172.21.15.97:0/3647633178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2b809f6e0 0x7ff2b811f5b0 unknown :-1 s=CLOSED pgs=12863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:42.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 >> 172.21.15.97:0/3647633178 conn(0x7ff2b80a1860 msgr2=0x7ff2b8134610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:42.200 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 shutdown_connections 2024-03-27T08:38:42.200 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.207+0000 7ff2be79d640 1 -- 172.21.15.97:0/3647633178 wait complete. 2024-03-27T08:38:42.210 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool application enable 97a8ed02-de19-41fe-985b-30a778c6d918 rados 2024-03-27T08:38:42.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.309+0000 7f0334a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:42.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.320+0000 7f0334a2d640 1 Processor -- start 2024-03-27T08:38:42.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.320+0000 7f0334a2d640 1 -- start start 2024-03-27T08:38:42.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.321+0000 7f0334a2d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0330137b10 0x7f0330139f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:42.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.321+0000 7f0334a2d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033006eb50 con 0x7f033009f6e0 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.321+0000 7f0334a2d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f033009f2f0 con 0x7f0330137b10 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.321+0000 7f0325d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0330137b10 0x7f0330139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.321+0000 7f032e59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f033009f6e0 0x7f033009faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35382/0 (socket says 172.21.15.97:35382) 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.321+0000 7f032e59c640 1 -- 172.21.15.97:0/892964160 learned_addr learned my addr 172.21.15.97:0/892964160 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f0325d9b640 1 -- 172.21.15.97:0/892964160 >> v1:172.21.15.97:6789/0 conn(0x7f033009f6e0 legacy=0x7f033009faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f032dd9b640 1 -- 172.21.15.97:0/892964160 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815026516 0 0) 0x7f033006eb50 con 0x7f033009f6e0 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f0325d9b640 1 -- 172.21.15.97:0/892964160 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03300a0200 con 0x7f0330137b10 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f0325d9b640 1 --2- 172.21.15.97:0/892964160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0330137b10 0x7f0330139f30 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1da1cb505c0b589 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f032dd9b640 1 -- 172.21.15.97:0/892964160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0318009d90 con 0x7f0330137b10 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f032dd9b640 1 -- 172.21.15.97:0/892964160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f031800e230 con 0x7f0330137b10 2024-03-27T08:38:42.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.322+0000 7f032dd9b640 1 -- 172.21.15.97:0/892964160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f031800e4d0 con 0x7f0330137b10 2024-03-27T08:38:42.315 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 -- 172.21.15.97:0/892964160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0330137b10 msgr2=0x7f0330139f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:42.315 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 --2- 172.21.15.97:0/892964160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0330137b10 0x7f0330139f30 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:42.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 -- 172.21.15.97:0/892964160 shutdown_connections 2024-03-27T08:38:42.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 --2- 172.21.15.97:0/892964160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0330137b10 0x7f0330139f30 unknown :-1 s=CLOSED pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:42.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 -- 172.21.15.97:0/892964160 >> 172.21.15.97:0/892964160 conn(0x7f03300a1860 msgr2=0x7f03300a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:42.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 -- 172.21.15.97:0/892964160 shutdown_connections 2024-03-27T08:38:42.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 1 -- 172.21.15.97:0/892964160 wait complete. 2024-03-27T08:38:42.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.323+0000 7f0334a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:42.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.326+0000 7f0334a2d640 1 Processor -- start 2024-03-27T08:38:42.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.326+0000 7f0334a2d640 1 -- start start 2024-03-27T08:38:42.319 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f0334a2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 0x7f033011f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:42.319 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f0334a2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0330120760 con 0x7f033009f6e0 2024-03-27T08:38:42.319 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f032e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 0x7f033011f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:42.319 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f032e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 0x7f033011f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50864/0 (socket says 172.21.15.97:50864) 2024-03-27T08:38:42.320 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f032e59c640 1 -- 172.21.15.97:0/3250771978 learned_addr learned my addr 172.21.15.97:0/3250771978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:42.320 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f032e59c640 1 -- 172.21.15.97:0/3250771978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033011f120 con 0x7f033009f6e0 2024-03-27T08:38:42.321 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f032e59c640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 0x7f033011f670 crc :-1 s=READY pgs=12866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:42.321 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.328+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0320004b80 con 0x7f033009f6e0 2024-03-27T08:38:42.321 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.329+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0320004ce0 con 0x7f033009f6e0 2024-03-27T08:38:42.321 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.329+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033011ed80 con 0x7f033009f6e0 2024-03-27T08:38:42.321 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.329+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0320007120 con 0x7f033009f6e0 2024-03-27T08:38:42.321 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.329+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0330122760 con 0x7f033009f6e0 2024-03-27T08:38:42.322 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.329+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f03200072d0 con 0x7f033009f6e0 2024-03-27T08:38:42.322 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.330+0000 7f03277fe640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f03100387a0 0x7f031003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:42.322 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.330+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033011ef70 con 0x7f033009f6e0 2024-03-27T08:38:42.322 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.330+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(686..686 src has 399..686) ==== 3778+0+0 (crc 0 0 0) 0x7f0320048590 con 0x7f033009f6e0 2024-03-27T08:38:42.322 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.330+0000 7f0325d9b640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f03100387a0 0x7f031003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:42.323 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.331+0000 7f0325d9b640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f03100387a0 0x7f031003acb0 crc :-1 s=READY pgs=3829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:42.327 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.336+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f032000f230 con 0x7f033009f6e0 2024-03-27T08:38:42.515 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:42.523+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "app": "rados"} v 0) -- 0x7f03300693c0 con 0x7f033009f6e0 2024-03-27T08:38:43.200 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:43.209+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "app": "rados"}]=0 enabled application 'rados' on pool '97a8ed02-de19-41fe-985b-30a778c6d918' v687) ==== 213+0+0 (crc 0 0 0) 0x7f032000a5d0 con 0x7f033009f6e0 2024-03-27T08:38:43.331 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:43.341+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "app": "rados"} v 0) -- 0x7f0330120fd0 con 0x7f033009f6e0 2024-03-27T08:38:44.206 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.214+0000 7f03277fe640 1 -- 172.21.15.97:0/3250771978 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "app": "rados"}]=0 enabled application 'rados' on pool '97a8ed02-de19-41fe-985b-30a778c6d918' v688) ==== 213+0+0 (crc 0 0 0) 0x7f032000a7b0 con 0x7f033009f6e0 2024-03-27T08:38:44.206 INFO:tasks.workunit.client.0.smithi097.stderr:enabled application 'rados' on pool '97a8ed02-de19-41fe-985b-30a778c6d918' 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f03100387a0 msgr2=0x7f031003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f03100387a0 0x7f031003acb0 crc :-1 s=READY pgs=3829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 msgr2=0x7f033011f670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 0x7f033011f670 crc :-1 s=READY pgs=12866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 shutdown_connections 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f03100387a0 0x7f031003acb0 unknown :-1 s=CLOSED pgs=3829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 --2- 172.21.15.97:0/3250771978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033009f6e0 0x7f033011f670 unknown :-1 s=CLOSED pgs=12866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 >> 172.21.15.97:0/3250771978 conn(0x7f03300a1860 msgr2=0x7f0330134610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 shutdown_connections 2024-03-27T08:38:44.209 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.217+0000 7f0334a2d640 1 -- 172.21.15.97:0/3250771978 wait complete. 2024-03-27T08:38:44.222 INFO:tasks.workunit.client.0.smithi097.stderr:++ seq 1 10 2024-03-27T08:38:44.311 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:44.311 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj1 /etc/passwd 2024-03-27T08:38:44.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.354+0000 7fc9e8cfaec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.351 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.359+0000 7fc9e8cfaec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.351 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.359+0000 7fc9e8cfaec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.352 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.360+0000 7fc9e8cfaec0 1 Processor -- start 2024-03-27T08:38:44.352 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.360+0000 7fc9e8cfaec0 1 -- start start 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e8cfaec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1a7eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e8cfaec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x19520b0 con 0x19e8c90 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e119b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1a7eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e119b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1a7eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50884/0 (socket says 172.21.15.97:50884) 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e119b640 1 -- 172.21.15.97:0/2148487765 learned_addr learned my addr 172.21.15.97:0/2148487765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e119b640 1 -- 172.21.15.97:0/2148487765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18f21e0 con 0x19e8c90 2024-03-27T08:38:44.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.361+0000 7fc9e119b640 1 --2- 172.21.15.97:0/2148487765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1a7eb40 crc :-1 s=READY pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c583f5eb9effb21 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9daffd640 1 -- 172.21.15.97:0/2148487765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc9cc029aa0 con 0x19e8c90 2024-03-27T08:38:44.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9daffd640 1 -- 172.21.15.97:0/2148487765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc9cc0312f0 con 0x19e8c90 2024-03-27T08:38:44.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9daffd640 1 -- 172.21.15.97:0/2148487765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc9cc031590 con 0x19e8c90 2024-03-27T08:38:44.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2148487765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 msgr2=0x1a7eb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2148487765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1a7eb40 crc :-1 s=READY pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2148487765 shutdown_connections 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2148487765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1a7eb40 unknown :-1 s=CLOSED pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2148487765 >> 172.21.15.97:0/2148487765 conn(0x1906ea0 msgr2=0x1a83060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2148487765 shutdown_connections 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.362+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2148487765 wait complete. 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.363+0000 7fc9e8cfaec0 1 Processor -- start 2024-03-27T08:38:44.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.363+0000 7fc9e8cfaec0 1 -- start start 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.364+0000 7fc9e8cfaec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1b0da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.364+0000 7fc9e8cfaec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x194fb30 con 0x19e8c90 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.364+0000 7fc9e119b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1b0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.364+0000 7fc9e119b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1b0da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50888/0 (socket says 172.21.15.97:50888) 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.364+0000 7fc9e119b640 1 -- 172.21.15.97:0/2848502473 learned_addr learned my addr 172.21.15.97:0/2848502473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.364+0000 7fc9e119b640 1 -- 172.21.15.97:0/2848502473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18e4e30 con 0x19e8c90 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.365+0000 7fc9e119b640 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1b0da40 crc :-1 s=READY pgs=12870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.360 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.365+0000 7fc9d9ffb640 1 -- 172.21.15.97:0/2848502473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc9cc036aa0 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.365+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x18e4890 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.365+0000 7fc9d9ffb640 1 -- 172.21.15.97:0/2848502473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc9cc036c00 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.365+0000 7fc9d9ffb640 1 -- 172.21.15.97:0/2848502473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc9cc036ea0 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.365+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1b0d480 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.366+0000 7fc9d9ffb640 1 -- 172.21.15.97:0/2848502473 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fc9cc031040 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.366+0000 7fc9d9ffb640 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc9c8036fd0 0x7fc9c80394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.366+0000 7fc9d9ffb640 1 -- 172.21.15.97:0/2848502473 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7fc9cc03b040 con 0x19e8c90 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.366+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> v2:172.21.15.190:6805/971408905 conn(0x19eb7f0 0x1b15d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.361 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.366+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 200.7 200:ff7b1f36:::obj1:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x1b162d0 con 0x19eb7f0 2024-03-27T08:38:44.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.366+0000 7fc9e099a640 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc9c8036fd0 0x7fc9c80394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.367+0000 7fc9e099a640 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc9c8036fd0 0x7fc9c80394e0 crc :-1 s=READY pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.367+0000 7fc9e199c640 1 --2- 172.21.15.97:0/2848502473 >> v2:172.21.15.190:6805/971408905 conn(0x19eb7f0 0x1b15d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:44.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.370+0000 7fc9e199c640 1 --2- 172.21.15.97:0/2848502473 >> v2:172.21.15.190:6805/971408905 conn(0x19eb7f0 0x1b15d90 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.616 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e199c640 1 -- 172.21.15.97:0/2848502473 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj1 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fc9d400c430 con 0x19eb7f0 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 >> v2:172.21.15.190:6805/971408905 conn(0x19eb7f0 msgr2=0x1b15d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> v2:172.21.15.190:6805/971408905 conn(0x19eb7f0 0x1b15d90 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc9c8036fd0 msgr2=0x7fc9c80394e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc9c8036fd0 0x7fc9c80394e0 crc :-1 s=READY pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 msgr2=0x1b0da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1b0da40 crc :-1 s=READY pgs=12870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.617 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 shutdown_connections 2024-03-27T08:38:44.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> v2:172.21.15.190:6805/971408905 conn(0x19eb7f0 0x1b15d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc9c8036fd0 0x7fc9c80394e0 unknown :-1 s=CLOSED pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 --2- 172.21.15.97:0/2848502473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x19e8c90 0x1b0da40 unknown :-1 s=CLOSED pgs=12870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 >> 172.21.15.97:0/2848502473 conn(0x1906ea0 msgr2=0x1a83060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 shutdown_connections 2024-03-27T08:38:44.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.623+0000 7fc9e8cfaec0 1 -- 172.21.15.97:0/2848502473 wait complete. 2024-03-27T08:38:44.619 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:44.619 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj2 /etc/passwd 2024-03-27T08:38:44.653 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.662+0000 7fe06962eec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.658 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.667+0000 7fe06962eec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.659 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.667+0000 7fe06962eec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.659 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.667+0000 7fe06962eec0 1 Processor -- start 2024-03-27T08:38:44.659 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.667+0000 7fe06962eec0 1 -- start start 2024-03-27T08:38:44.661 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.668+0000 7fe06962eec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x239e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.661 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.668+0000 7fe06962eec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x230a0b0 con 0x23a0c90 2024-03-27T08:38:44.661 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.668+0000 7fe06179b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x239e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe06179b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x239e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50912/0 (socket says 172.21.15.97:50912) 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe06179b640 1 -- 172.21.15.97:0/1365042800 learned_addr learned my addr 172.21.15.97:0/1365042800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe06179b640 1 -- 172.21.15.97:0/1365042800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x22aa1e0 con 0x23a0c90 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe06179b640 1 --2- 172.21.15.97:0/1365042800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x239e8c0 crc :-1 s=READY pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230ef051474b91ac server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe05affd640 1 -- 172.21.15.97:0/1365042800 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe054029d10 con 0x23a0c90 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe05affd640 1 -- 172.21.15.97:0/1365042800 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe05400a040 con 0x23a0c90 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.669+0000 7fe05affd640 1 -- 172.21.15.97:0/1365042800 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe05400a2e0 con 0x23a0c90 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 -- 172.21.15.97:0/1365042800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 msgr2=0x239e8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 --2- 172.21.15.97:0/1365042800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x239e8c0 crc :-1 s=READY pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 -- 172.21.15.97:0/1365042800 shutdown_connections 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 --2- 172.21.15.97:0/1365042800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x239e8c0 unknown :-1 s=CLOSED pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.662 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 -- 172.21.15.97:0/1365042800 >> 172.21.15.97:0/1365042800 conn(0x22beea0 msgr2=0x243b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 -- 172.21.15.97:0/1365042800 shutdown_connections 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 -- 172.21.15.97:0/1365042800 wait complete. 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.670+0000 7fe06962eec0 1 Processor -- start 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.671+0000 7fe06962eec0 1 -- start start 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06962eec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x24c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06962eec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x2307b30 con 0x23a0c90 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06179b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x24c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.665 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06179b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x24c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50916/0 (socket says 172.21.15.97:50916) 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06179b640 1 -- 172.21.15.97:0/365884775 learned_addr learned my addr 172.21.15.97:0/365884775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06179b640 1 -- 172.21.15.97:0/365884775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x229ce30 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06179b640 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x24c4860 crc :-1 s=READY pgs=12874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe059ffb640 1 -- 172.21.15.97:0/365884775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe054006a20 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe059ffb640 1 -- 172.21.15.97:0/365884775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe054006b80 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x229c890 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe059ffb640 1 -- 172.21.15.97:0/365884775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe054006e20 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.672+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x24c5840 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.673+0000 7fe059ffb640 1 -- 172.21.15.97:0/365884775 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe054031420 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.673+0000 7fe059ffb640 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe048037030 0x7fe048039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.673+0000 7fe059ffb640 1 -- 172.21.15.97:0/365884775 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7fe054038040 con 0x23a0c90 2024-03-27T08:38:44.666 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.674+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> v2:172.21.15.190:6805/971408905 conn(0x23a37f0 0x24d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.667 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.674+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 200.8 200:104778fc:::obj2:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x24d1290 con 0x23a37f0 2024-03-27T08:38:44.667 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.674+0000 7fe060f9a640 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe048037030 0x7fe048039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.667 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.674+0000 7fe060f9a640 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe048037030 0x7fe048039540 crc :-1 s=READY pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.667 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.674+0000 7fe061f9c640 1 --2- 172.21.15.97:0/365884775 >> v2:172.21.15.190:6805/971408905 conn(0x23a37f0 0x24d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:44.668 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.677+0000 7fe061f9c640 1 --2- 172.21.15.97:0/365884775 >> v2:172.21.15.190:6805/971408905 conn(0x23a37f0 0x24d0d50 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe061f9c640 1 -- 172.21.15.97:0/365884775 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj2 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fe04c00c430 con 0x23a37f0 2024-03-27T08:38:44.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 >> v2:172.21.15.190:6805/971408905 conn(0x23a37f0 msgr2=0x24d0d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> v2:172.21.15.190:6805/971408905 conn(0x23a37f0 0x24d0d50 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe048037030 msgr2=0x7fe048039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe048037030 0x7fe048039540 crc :-1 s=READY pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 msgr2=0x24c4860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x24c4860 crc :-1 s=READY pgs=12874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 shutdown_connections 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> v2:172.21.15.190:6805/971408905 conn(0x23a37f0 0x24d0d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe048037030 0x7fe048039540 unknown :-1 s=CLOSED pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 --2- 172.21.15.97:0/365884775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x23a0c90 0x24c4860 unknown :-1 s=CLOSED pgs=12874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 >> 172.21.15.97:0/365884775 conn(0x22beea0 msgr2=0x243b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 shutdown_connections 2024-03-27T08:38:44.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.687+0000 7fe06962eec0 1 -- 172.21.15.97:0/365884775 wait complete. 2024-03-27T08:38:44.683 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:44.683 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj3 /etc/passwd 2024-03-27T08:38:44.716 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.725+0000 7fe9b003aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.721 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.730+0000 7fe9b003aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.721 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.730+0000 7fe9b003aec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.722 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.730+0000 7fe9b003aec0 1 Processor -- start 2024-03-27T08:38:44.722 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.730+0000 7fe9b003aec0 1 -- start start 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.731+0000 7fe9b003aec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.731+0000 7fe9b003aec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1dfa0b0 con 0x1e7a8f0 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.731+0000 7fe9a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.731+0000 7fe9a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fb7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50936/0 (socket says 172.21.15.97:50936) 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.732+0000 7fe9a37fe640 1 -- 172.21.15.97:0/1137483966 learned_addr learned my addr 172.21.15.97:0/1137483966 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.732+0000 7fe9a37fe640 1 -- 172.21.15.97:0/1137483966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1d9a1e0 con 0x1e7a8f0 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.732+0000 7fe9a37fe640 1 --2- 172.21.15.97:0/1137483966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fb7a00 crc :-1 s=READY pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a40f3041329bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.732+0000 7fe9a1ffb640 1 -- 172.21.15.97:0/1137483966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe99c00b5d0 con 0x1e7a8f0 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.732+0000 7fe9a1ffb640 1 -- 172.21.15.97:0/1137483966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe99c00b730 con 0x1e7a8f0 2024-03-27T08:38:44.724 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.732+0000 7fe9a1ffb640 1 -- 172.21.15.97:0/1137483966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe99c00b9d0 con 0x1e7a8f0 2024-03-27T08:38:44.725 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 -- 172.21.15.97:0/1137483966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 msgr2=0x1fb7a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.725 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/1137483966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fb7a00 crc :-1 s=READY pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 -- 172.21.15.97:0/1137483966 shutdown_connections 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/1137483966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fb7a00 unknown :-1 s=CLOSED pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 -- 172.21.15.97:0/1137483966 >> 172.21.15.97:0/1137483966 conn(0x1daeea0 msgr2=0x1f27ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 -- 172.21.15.97:0/1137483966 shutdown_connections 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 -- 172.21.15.97:0/1137483966 wait complete. 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 Processor -- start 2024-03-27T08:38:44.726 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.733+0000 7fe9b003aec0 1 -- start start 2024-03-27T08:38:44.727 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.734+0000 7fe9b003aec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fbf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.727 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.734+0000 7fe9b003aec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1df7b30 con 0x1e7a8f0 2024-03-27T08:38:44.727 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.734+0000 7fe9a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fbf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fbf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50938/0 (socket says 172.21.15.97:50938) 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a37fe640 1 -- 172.21.15.97:0/664551010 learned_addr learned my addr 172.21.15.97:0/664551010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a37fe640 1 -- 172.21.15.97:0/664551010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1d8ce30 con 0x1e7a8f0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a37fe640 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fbf010 crc :-1 s=READY pgs=12878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a0ff9640 1 -- 172.21.15.97:0/664551010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe99c0129e0 con 0x1e7a8f0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a0ff9640 1 -- 172.21.15.97:0/664551010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe99c005040 con 0x1e7a8f0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1d8c890 con 0x1e7a8f0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9a0ff9640 1 -- 172.21.15.97:0/664551010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe99c012c70 con 0x1e7a8f0 2024-03-27T08:38:44.728 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.735+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1f10700 con 0x1e7a8f0 2024-03-27T08:38:44.729 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.736+0000 7fe9a0ff9640 1 -- 172.21.15.97:0/664551010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fe99c00d040 con 0x1e7a8f0 2024-03-27T08:38:44.729 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.736+0000 7fe9a0ff9640 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe984036fe0 0x7fe9840394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.729 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.736+0000 7fe9a0ff9640 1 -- 172.21.15.97:0/664551010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7fe99c00a040 con 0x1e7a8f0 2024-03-27T08:38:44.729 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.736+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> v2:172.21.15.97:6805/1668842645 conn(0x1e7b410 0x1fb40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.729 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.736+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 200.1 200:8dd16f86:::obj3:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x1fb4620 con 0x1e7b410 2024-03-27T08:38:44.730 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.737+0000 7fe9a2ffd640 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe984036fe0 0x7fe9840394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.730 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.737+0000 7fe9a2ffd640 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe984036fe0 0x7fe9840394f0 crc :-1 s=READY pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.730 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.737+0000 7fe9a899c640 1 --2- 172.21.15.97:0/664551010 >> v2:172.21.15.97:6805/1668842645 conn(0x1e7b410 0x1fb40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:44.732 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.741+0000 7fe9a899c640 1 --2- 172.21.15.97:0/664551010 >> v2:172.21.15.97:6805/1668842645 conn(0x1e7b410 0x1fb40e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.919 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9a899c640 1 -- 172.21.15.97:0/664551010 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj3 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fe99400a430 con 0x1e7b410 2024-03-27T08:38:44.919 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 >> v2:172.21.15.97:6805/1668842645 conn(0x1e7b410 msgr2=0x1fb40e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.919 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> v2:172.21.15.97:6805/1668842645 conn(0x1e7b410 0x1fb40e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.919 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe984036fe0 msgr2=0x7fe9840394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe984036fe0 0x7fe9840394f0 crc :-1 s=READY pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 msgr2=0x1fbf010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fbf010 crc :-1 s=READY pgs=12878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 shutdown_connections 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> v2:172.21.15.97:6805/1668842645 conn(0x1e7b410 0x1fb40e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fe984036fe0 0x7fe9840394f0 unknown :-1 s=CLOSED pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 --2- 172.21.15.97:0/664551010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1e7a8f0 0x1fbf010 unknown :-1 s=CLOSED pgs=12878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 >> 172.21.15.97:0/664551010 conn(0x1daeea0 msgr2=0x1f27ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.926+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 shutdown_connections 2024-03-27T08:38:44.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.927+0000 7fe9b003aec0 1 -- 172.21.15.97:0/664551010 wait complete. 2024-03-27T08:38:44.921 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:44.921 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj4 /etc/passwd 2024-03-27T08:38:44.956 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.963+0000 7f9eec0faec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.960 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.968+0000 7f9eec0faec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.960 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.968+0000 7f9eec0faec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:44.960 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.969+0000 7f9eec0faec0 1 Processor -- start 2024-03-27T08:38:44.961 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.969+0000 7f9eec0faec0 1 -- start start 2024-03-27T08:38:44.964 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.970+0000 7f9eec0faec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.964 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.970+0000 7f9eec0faec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xbdc0b0 con 0xc72c90 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.970+0000 7f9edf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.970+0000 7f9edf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd056c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50958/0 (socket says 172.21.15.97:50958) 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.970+0000 7f9edf7fe640 1 -- 172.21.15.97:0/2935466131 learned_addr learned my addr 172.21.15.97:0/2935466131 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9edf7fe640 1 -- 172.21.15.97:0/2935466131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb7c1e0 con 0xc72c90 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9edf7fe640 1 --2- 172.21.15.97:0/2935466131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd056c0 crc :-1 s=READY pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92175f253073d117 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eddffb640 1 -- 172.21.15.97:0/2935466131 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9ed00061f0 con 0xc72c90 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eddffb640 1 -- 172.21.15.97:0/2935466131 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9ed00019f0 con 0xc72c90 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eddffb640 1 -- 172.21.15.97:0/2935466131 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9ed0001b60 con 0xc72c90 2024-03-27T08:38:44.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 -- 172.21.15.97:0/2935466131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 msgr2=0xd056c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/2935466131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd056c0 crc :-1 s=READY pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 -- 172.21.15.97:0/2935466131 shutdown_connections 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/2935466131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd056c0 unknown :-1 s=CLOSED pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 -- 172.21.15.97:0/2935466131 >> 172.21.15.97:0/2935466131 conn(0xb90ea0 msgr2=0xd08ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 -- 172.21.15.97:0/2935466131 shutdown_connections 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.971+0000 7f9eec0faec0 1 -- 172.21.15.97:0/2935466131 wait complete. 2024-03-27T08:38:44.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.972+0000 7f9eec0faec0 1 Processor -- start 2024-03-27T08:38:44.967 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.972+0000 7f9eec0faec0 1 -- start start 2024-03-27T08:38:44.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.973+0000 7f9eec0faec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd05a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.973+0000 7f9eec0faec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xbd9b30 con 0xc72c90 2024-03-27T08:38:44.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.973+0000 7f9edf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.973+0000 7f9edf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd05a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50970/0 (socket says 172.21.15.97:50970) 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.973+0000 7f9edf7fe640 1 -- 172.21.15.97:0/1469826570 learned_addr learned my addr 172.21.15.97:0/1469826570 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9edf7fe640 1 -- 172.21.15.97:0/1469826570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb6ee30 con 0xc72c90 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9edf7fe640 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd05a10 crc :-1 s=READY pgs=12882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9edcff9640 1 -- 172.21.15.97:0/1469826570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9ed00049d0 con 0xc72c90 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9edcff9640 1 -- 172.21.15.97:0/1469826570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9ed0004b30 con 0xc72c90 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9edcff9640 1 -- 172.21.15.97:0/1469826570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9ed0004dd0 con 0xc72c90 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xb6e890 con 0xc72c90 2024-03-27T08:38:44.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.974+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0xc715b0 con 0xc72c90 2024-03-27T08:38:44.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.975+0000 7f9edcff9640 1 -- 172.21.15.97:0/1469826570 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f9ed00133a0 con 0xc72c90 2024-03-27T08:38:44.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.975+0000 7f9edcff9640 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f9ecc037030 0x7f9ecc039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.975+0000 7f9edcff9640 1 -- 172.21.15.97:0/1469826570 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7f9ed00483d0 con 0xc72c90 2024-03-27T08:38:44.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.975+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> v2:172.21.15.190:6805/971408905 conn(0xc74880 0xc5ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:44.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.975+0000 7f9edeffd640 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f9ecc037030 0x7f9ecc039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:44.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.975+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 200.0 200:0ee9ae15:::obj4:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0xd98180 con 0xc74880 2024-03-27T08:38:44.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.976+0000 7f9edeffd640 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f9ecc037030 0x7f9ecc039540 crc :-1 s=READY pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.976+0000 7f9ee4d9c640 1 --2- 172.21.15.97:0/1469826570 >> v2:172.21.15.190:6805/971408905 conn(0xc74880 0xc5ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:44.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.979+0000 7f9ee4d9c640 1 --2- 172.21.15.97:0/1469826570 >> v2:172.21.15.190:6805/971408905 conn(0xc74880 0xc5ce00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:44.983 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9ee4d9c640 1 -- 172.21.15.97:0/1469826570 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj4 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9ed800c430 con 0xc74880 2024-03-27T08:38:44.983 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 >> v2:172.21.15.190:6805/971408905 conn(0xc74880 msgr2=0xc5ce00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.983 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> v2:172.21.15.190:6805/971408905 conn(0xc74880 0xc5ce00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f9ecc037030 msgr2=0x7f9ecc039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f9ecc037030 0x7f9ecc039540 crc :-1 s=READY pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 msgr2=0xd05a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd05a10 crc :-1 s=READY pgs=12882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 shutdown_connections 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> v2:172.21.15.190:6805/971408905 conn(0xc74880 0xc5ce00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f9ecc037030 0x7f9ecc039540 unknown :-1 s=CLOSED pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 --2- 172.21.15.97:0/1469826570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xc72c90 0xd05a10 unknown :-1 s=CLOSED pgs=12882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 >> 172.21.15.97:0/1469826570 conn(0xb90ea0 msgr2=0xd0c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 shutdown_connections 2024-03-27T08:38:44.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:44.991+0000 7f9eec0faec0 1 -- 172.21.15.97:0/1469826570 wait complete. 2024-03-27T08:38:44.985 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:44.985 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj5 /etc/passwd 2024-03-27T08:38:45.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.027+0000 7f0cb8a59ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.032+0000 7f0cb8a59ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.032+0000 7f0cb8a59ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.033+0000 7f0cb8a59ec0 1 Processor -- start 2024-03-27T08:38:45.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.033+0000 7f0cb8a59ec0 1 -- start start 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.034+0000 7f0cb8a59ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x1025810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.034+0000 7f0cb8a59ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xf870b0 con 0x101dc90 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.034+0000 7f0cb0b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x1025810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.034+0000 7f0cb0b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x1025810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50978/0 (socket says 172.21.15.97:50978) 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.034+0000 7f0cb0b9b640 1 -- 172.21.15.97:0/3164775842 learned_addr learned my addr 172.21.15.97:0/3164775842 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.034+0000 7f0cb0b9b640 1 -- 172.21.15.97:0/3164775842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xf271e0 con 0x101dc90 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb0b9b640 1 --2- 172.21.15.97:0/3164775842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x1025810 crc :-1 s=READY pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486b7d99f0596d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0caa7fc640 1 -- 172.21.15.97:0/3164775842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0c9c0312f0 con 0x101dc90 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0caa7fc640 1 -- 172.21.15.97:0/3164775842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0c9c031450 con 0x101dc90 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0caa7fc640 1 -- 172.21.15.97:0/3164775842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0c9c0316f0 con 0x101dc90 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3164775842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 msgr2=0x1025810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.032 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3164775842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x1025810 crc :-1 s=READY pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3164775842 shutdown_connections 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3164775842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x1025810 unknown :-1 s=CLOSED pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3164775842 >> 172.21.15.97:0/3164775842 conn(0xf3bea0 msgr2=0x10b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3164775842 shutdown_connections 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.035+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3164775842 wait complete. 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.036+0000 7f0cb8a59ec0 1 Processor -- start 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.036+0000 7f0cb8a59ec0 1 -- start start 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.037+0000 7f0cb8a59ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x114c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.037+0000 7f0cb8a59ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xf84b30 con 0x101dc90 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.037+0000 7f0cb0b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x114c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.037+0000 7f0cb0b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x114c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50982/0 (socket says 172.21.15.97:50982) 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.037+0000 7f0cb0b9b640 1 -- 172.21.15.97:0/3655454624 learned_addr learned my addr 172.21.15.97:0/3655454624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0cb0b9b640 1 -- 172.21.15.97:0/3655454624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xf19e30 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0cb0b9b640 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x114c180 crc :-1 s=READY pgs=12886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0ca97fa640 1 -- 172.21.15.97:0/3655454624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0c9c031c30 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0ca97fa640 1 -- 172.21.15.97:0/3655454624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0c9c031d90 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xf19890 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0ca97fa640 1 -- 172.21.15.97:0/3655454624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0c9c034440 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.038+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x114bcc0 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.039+0000 7f0ca97fa640 1 -- 172.21.15.97:0/3655454624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f0c9c031040 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.039+0000 7f0ca97fa640 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0c98037030 0x7f0c98039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.039+0000 7f0ca97fa640 1 -- 172.21.15.97:0/3655454624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7f0c9c03a040 con 0x101dc90 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.039+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> v2:172.21.15.190:6805/971408905 conn(0x10207f0 0x10078f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.039+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 200.3 200:c52c9666:::obj5:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x1007e30 con 0x10207f0 2024-03-27T08:38:45.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.039+0000 7f0cab7fe640 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0c98037030 0x7f0c98039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.035 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.040+0000 7f0cab7fe640 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0c98037030 0x7f0c98039540 crc :-1 s=READY pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.035 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.040+0000 7f0cb139c640 1 --2- 172.21.15.97:0/3655454624 >> v2:172.21.15.190:6805/971408905 conn(0x10207f0 0x10078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:45.036 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.042+0000 7f0cb139c640 1 --2- 172.21.15.97:0/3655454624 >> v2:172.21.15.190:6805/971408905 conn(0x10207f0 0x10078f0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.046 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb139c640 1 -- 172.21.15.97:0/3655454624 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj5 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f0ca400c430 con 0x10207f0 2024-03-27T08:38:45.046 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 >> v2:172.21.15.190:6805/971408905 conn(0x10207f0 msgr2=0x10078f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.046 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> v2:172.21.15.190:6805/971408905 conn(0x10207f0 0x10078f0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.046 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0c98037030 msgr2=0x7f0c98039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.046 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0c98037030 0x7f0c98039540 crc :-1 s=READY pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.046 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 msgr2=0x114c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x114c180 crc :-1 s=READY pgs=12886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 shutdown_connections 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> v2:172.21.15.190:6805/971408905 conn(0x10207f0 0x10078f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0c98037030 0x7f0c98039540 unknown :-1 s=CLOSED pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 --2- 172.21.15.97:0/3655454624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x101dc90 0x114c180 unknown :-1 s=CLOSED pgs=12886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 >> 172.21.15.97:0/3655454624 conn(0xf3bea0 msgr2=0x10b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 shutdown_connections 2024-03-27T08:38:45.047 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.053+0000 7f0cb8a59ec0 1 -- 172.21.15.97:0/3655454624 wait complete. 2024-03-27T08:38:45.048 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:45.048 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj6 /etc/passwd 2024-03-27T08:38:45.082 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.090+0000 7f7755662ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.087 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.095+0000 7f7755662ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.087 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.095+0000 7f7755662ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.088 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.096+0000 7f7755662ec0 1 Processor -- start 2024-03-27T08:38:45.088 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.096+0000 7f7755662ec0 1 -- start start 2024-03-27T08:38:45.090 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f7755662ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x123e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.090 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f7755662ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x110e0b0 con 0x118f000 2024-03-27T08:38:45.090 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f774d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x123e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f774d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x123e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50998/0 (socket says 172.21.15.97:50998) 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f774d79b640 1 -- 172.21.15.97:0/769086007 learned_addr learned my addr 172.21.15.97:0/769086007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f774d79b640 1 -- 172.21.15.97:0/769086007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x10ae1e0 con 0x118f000 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f774d79b640 1 --2- 172.21.15.97:0/769086007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x123e800 crc :-1 s=READY pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b4af4491ed6d70 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f7746ffd640 1 -- 172.21.15.97:0/769086007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f773800b5d0 con 0x118f000 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f7746ffd640 1 -- 172.21.15.97:0/769086007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f773800b730 con 0x118f000 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.097+0000 7f7746ffd640 1 -- 172.21.15.97:0/769086007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f773800b9d0 con 0x118f000 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 -- 172.21.15.97:0/769086007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 msgr2=0x123e800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 --2- 172.21.15.97:0/769086007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x123e800 crc :-1 s=READY pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 -- 172.21.15.97:0/769086007 shutdown_connections 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 --2- 172.21.15.97:0/769086007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x123e800 unknown :-1 s=CLOSED pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 -- 172.21.15.97:0/769086007 >> 172.21.15.97:0/769086007 conn(0x10c2ea0 msgr2=0x11a3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 -- 172.21.15.97:0/769086007 shutdown_connections 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.098+0000 7f7755662ec0 1 -- 172.21.15.97:0/769086007 wait complete. 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.099+0000 7f7755662ec0 1 Processor -- start 2024-03-27T08:38:45.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.099+0000 7f7755662ec0 1 -- start start 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f7755662ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x12d3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f7755662ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x110bb30 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f774d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x12d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f774d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x12d3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51006/0 (socket says 172.21.15.97:51006) 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f774d79b640 1 -- 172.21.15.97:0/2870970827 learned_addr learned my addr 172.21.15.97:0/2870970827 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f774d79b640 1 -- 172.21.15.97:0/2870970827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x10a0e30 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.100+0000 7f774d79b640 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x12d3300 crc :-1 s=READY pgs=12890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.101+0000 7f7745ffb640 1 -- 172.21.15.97:0/2870970827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7738012890 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.101+0000 7f7745ffb640 1 -- 172.21.15.97:0/2870970827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7738005040 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.101+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x10a0890 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.101+0000 7f7745ffb640 1 -- 172.21.15.97:0/2870970827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7738012b20 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.101+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x12cc040 con 0x118f000 2024-03-27T08:38:45.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.101+0000 7f7745ffb640 1 -- 172.21.15.97:0/2870970827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f773800d040 con 0x118f000 2024-03-27T08:38:45.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.102+0000 7f7745ffb640 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7734037030 0x7f7734039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.102+0000 7f7745ffb640 1 -- 172.21.15.97:0/2870970827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7f773800a040 con 0x118f000 2024-03-27T08:38:45.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.102+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> v2:172.21.15.97:6805/1668842645 conn(0x11a9230 0x123e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.102+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 200.4 200:233a42c1:::obj6:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x1229b50 con 0x11a9230 2024-03-27T08:38:45.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.102+0000 7f774cf9a640 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7734037030 0x7f7734039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.097 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.102+0000 7f774cf9a640 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7734037030 0x7f7734039540 crc :-1 s=READY pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.097 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.103+0000 7f774df9c640 1 --2- 172.21.15.97:0/2870970827 >> v2:172.21.15.97:6805/1668842645 conn(0x11a9230 0x123e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:45.101 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.110+0000 7f774df9c640 1 --2- 172.21.15.97:0/2870970827 >> v2:172.21.15.97:6805/1668842645 conn(0x11a9230 0x123e6c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.202 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f774df9c640 1 -- 172.21.15.97:0/2870970827 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj6 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f774000a430 con 0x11a9230 2024-03-27T08:38:45.202 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 >> v2:172.21.15.97:6805/1668842645 conn(0x11a9230 msgr2=0x123e6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> v2:172.21.15.97:6805/1668842645 conn(0x11a9230 0x123e6c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7734037030 msgr2=0x7f7734039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7734037030 0x7f7734039540 crc :-1 s=READY pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 msgr2=0x12d3300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x12d3300 crc :-1 s=READY pgs=12890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 shutdown_connections 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> v2:172.21.15.97:6805/1668842645 conn(0x11a9230 0x123e6c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7734037030 0x7f7734039540 unknown :-1 s=CLOSED pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 --2- 172.21.15.97:0/2870970827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x118f000 0x12d3300 unknown :-1 s=CLOSED pgs=12890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.209+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 >> 172.21.15.97:0/2870970827 conn(0x10c2ea0 msgr2=0x11a3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.210+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 shutdown_connections 2024-03-27T08:38:45.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.210+0000 7f7755662ec0 1 -- 172.21.15.97:0/2870970827 wait complete. 2024-03-27T08:38:45.204 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:45.205 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj7 /etc/passwd 2024-03-27T08:38:45.241 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.250+0000 7f48cea31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.255+0000 7f48cea31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.255+0000 7f48cea31ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.256+0000 7f48cea31ec0 1 Processor -- start 2024-03-27T08:38:45.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.256+0000 7f48cea31ec0 1 -- start start 2024-03-27T08:38:45.251 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.257+0000 7f48cea31ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x20b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.257+0000 7f48cea31ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x201c0b0 con 0x209c8f0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.257+0000 7f48c639a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x20b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.257+0000 7f48c639a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x20b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51032/0 (socket says 172.21.15.97:51032) 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.257+0000 7f48c639a640 1 -- 172.21.15.97:0/3269496708 learned_addr learned my addr 172.21.15.97:0/3269496708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.258+0000 7f48c639a640 1 -- 172.21.15.97:0/3269496708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1fbc1e0 con 0x209c8f0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.258+0000 7f48c639a640 1 --2- 172.21.15.97:0/3269496708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x20b11e0 crc :-1 s=READY pgs=12893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825f6b1903651180 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.258+0000 7f48c4b97640 1 -- 172.21.15.97:0/3269496708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f48bc029a90 con 0x209c8f0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.258+0000 7f48c4b97640 1 -- 172.21.15.97:0/3269496708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f48bc029bf0 con 0x209c8f0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.258+0000 7f48c4b97640 1 -- 172.21.15.97:0/3269496708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f48bc006790 con 0x209c8f0 2024-03-27T08:38:45.252 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 -- 172.21.15.97:0/3269496708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 msgr2=0x20b11e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/3269496708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x20b11e0 crc :-1 s=READY pgs=12893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 -- 172.21.15.97:0/3269496708 shutdown_connections 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/3269496708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x20b11e0 unknown :-1 s=CLOSED pgs=12893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 -- 172.21.15.97:0/3269496708 >> 172.21.15.97:0/3269496708 conn(0x1fd0ea0 msgr2=0x21468e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 -- 172.21.15.97:0/3269496708 shutdown_connections 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.259+0000 7f48cea31ec0 1 -- 172.21.15.97:0/3269496708 wait complete. 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.260+0000 7f48cea31ec0 1 Processor -- start 2024-03-27T08:38:45.253 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.260+0000 7f48cea31ec0 1 -- start start 2024-03-27T08:38:45.256 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48cea31ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x21db840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.256 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48cea31ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x2019b30 con 0x209c8f0 2024-03-27T08:38:45.256 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48c639a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x21db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.256 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48c639a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x21db840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51038/0 (socket says 172.21.15.97:51038) 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48c639a640 1 -- 172.21.15.97:0/1997822381 learned_addr learned my addr 172.21.15.97:0/1997822381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48c639a640 1 -- 172.21.15.97:0/1997822381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1faee30 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.261+0000 7f48c639a640 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x21db840 crc :-1 s=READY pgs=12894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.262+0000 7f48b77fe640 1 -- 172.21.15.97:0/1997822381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f48bc03a040 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.262+0000 7f48b77fe640 1 -- 172.21.15.97:0/1997822381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f48bc006e70 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.262+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1fae890 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.262+0000 7f48b77fe640 1 -- 172.21.15.97:0/1997822381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f48bc035580 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.262+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2148af0 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.262+0000 7f48b77fe640 1 -- 172.21.15.97:0/1997822381 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f48bc03d040 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.263+0000 7f48b77fe640 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f48a4037030 0x7f48a4039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.263+0000 7f48b77fe640 1 -- 172.21.15.97:0/1997822381 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7f48bc02e040 con 0x209c8f0 2024-03-27T08:38:45.257 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.263+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> v2:172.21.15.97:6801/3203341735 conn(0x209d240 0x21e3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.258 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.263+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 --> v2:172.21.15.97:6801/3203341735 -- osd_op(unknown.0.0:1 200.9 200:9a09113a:::obj7:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x21dac10 con 0x209d240 2024-03-27T08:38:45.258 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.263+0000 7f48c5b99640 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f48a4037030 0x7f48a4039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.258 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.264+0000 7f48c5b99640 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f48a4037030 0x7f48a4039540 crc :-1 s=READY pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.258 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.264+0000 7f48c6b9b640 1 --2- 172.21.15.97:0/1997822381 >> v2:172.21.15.97:6801/3203341735 conn(0x209d240 0x21e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:45.259 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.267+0000 7f48c6b9b640 1 --2- 172.21.15.97:0/1997822381 >> v2:172.21.15.97:6801/3203341735 conn(0x209d240 0x21e3400 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48c6b9b640 1 -- 172.21.15.97:0/1997822381 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== osd_op_reply(1 obj7 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f48b800a430 con 0x209d240 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 >> v2:172.21.15.97:6801/3203341735 conn(0x209d240 msgr2=0x21e3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> v2:172.21.15.97:6801/3203341735 conn(0x209d240 0x21e3400 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f48a4037030 msgr2=0x7f48a4039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f48a4037030 0x7f48a4039540 crc :-1 s=READY pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 msgr2=0x21db840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x21db840 crc :-1 s=READY pgs=12894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 shutdown_connections 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> v2:172.21.15.97:6801/3203341735 conn(0x209d240 0x21e3400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f48a4037030 0x7f48a4039540 unknown :-1 s=CLOSED pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 --2- 172.21.15.97:0/1997822381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x209c8f0 0x21db840 unknown :-1 s=CLOSED pgs=12894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 >> 172.21.15.97:0/1997822381 conn(0x1fd0ea0 msgr2=0x214f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 shutdown_connections 2024-03-27T08:38:45.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.375+0000 7f48cea31ec0 1 -- 172.21.15.97:0/1997822381 wait complete. 2024-03-27T08:38:45.369 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:45.369 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj8 /etc/passwd 2024-03-27T08:38:45.404 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.412+0000 7faa9a036ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.417+0000 7faa9a036ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.417+0000 7faa9a036ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.410 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.418+0000 7faa9a036ec0 1 Processor -- start 2024-03-27T08:38:45.410 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.418+0000 7faa9a036ec0 1 -- start start 2024-03-27T08:38:45.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.419+0000 7faa9a036ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x219f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.419+0000 7faa9a036ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1fec0b0 con 0x2086ad0 2024-03-27T08:38:45.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.419+0000 7faa9219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x219f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.419+0000 7faa9219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x219f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51052/0 (socket says 172.21.15.97:51052) 2024-03-27T08:38:45.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.419+0000 7faa9219b640 1 -- 172.21.15.97:0/3341256643 learned_addr learned my addr 172.21.15.97:0/3341256643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.412 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.420+0000 7faa9219b640 1 -- 172.21.15.97:0/3341256643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1f8c1e0 con 0x2086ad0 2024-03-27T08:38:45.412 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.420+0000 7faa9219b640 1 --2- 172.21.15.97:0/3341256643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x219f890 crc :-1 s=READY pgs=12897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28fdf247cc8d1a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.412 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.420+0000 7faa90998640 1 -- 172.21.15.97:0/3341256643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7faa7c02e960 con 0x2086ad0 2024-03-27T08:38:45.412 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.420+0000 7faa90998640 1 -- 172.21.15.97:0/3341256643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7faa7c02eac0 con 0x2086ad0 2024-03-27T08:38:45.412 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.420+0000 7faa90998640 1 -- 172.21.15.97:0/3341256643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7faa7c02ed60 con 0x2086ad0 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 -- 172.21.15.97:0/3341256643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 msgr2=0x219f890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/3341256643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x219f890 crc :-1 s=READY pgs=12897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 -- 172.21.15.97:0/3341256643 shutdown_connections 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/3341256643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x219f890 unknown :-1 s=CLOSED pgs=12897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 -- 172.21.15.97:0/3341256643 >> 172.21.15.97:0/3341256643 conn(0x1fa0ea0 msgr2=0x21a29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 -- 172.21.15.97:0/3341256643 shutdown_connections 2024-03-27T08:38:45.414 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.422+0000 7faa9a036ec0 1 -- 172.21.15.97:0/3341256643 wait complete. 2024-03-27T08:38:45.415 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.423+0000 7faa9a036ec0 1 Processor -- start 2024-03-27T08:38:45.415 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.423+0000 7faa9a036ec0 1 -- start start 2024-03-27T08:38:45.417 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.424+0000 7faa9a036ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x2118400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.417 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.424+0000 7faa9a036ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1fe9b30 con 0x2086ad0 2024-03-27T08:38:45.417 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x2118400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.417 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x2118400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51056/0 (socket says 172.21.15.97:51056) 2024-03-27T08:38:45.417 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9219b640 1 -- 172.21.15.97:0/2058688572 learned_addr learned my addr 172.21.15.97:0/2058688572 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.417 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9219b640 1 -- 172.21.15.97:0/2058688572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1f7ee30 con 0x2086ad0 2024-03-27T08:38:45.418 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9219b640 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x2118400 crc :-1 s=READY pgs=12898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.418 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa8affd640 1 -- 172.21.15.97:0/2058688572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7faa7c02ec30 con 0x2086ad0 2024-03-27T08:38:45.418 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa8affd640 1 -- 172.21.15.97:0/2058688572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7faa7c02ed90 con 0x2086ad0 2024-03-27T08:38:45.418 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1f7e890 con 0x2086ad0 2024-03-27T08:38:45.418 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa8affd640 1 -- 172.21.15.97:0/2058688572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7faa7c032510 con 0x2086ad0 2024-03-27T08:38:45.418 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.425+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2105fc0 con 0x2086ad0 2024-03-27T08:38:45.419 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.426+0000 7faa8affd640 1 -- 172.21.15.97:0/2058688572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7faa7c03b040 con 0x2086ad0 2024-03-27T08:38:45.419 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.426+0000 7faa8affd640 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7faa78037030 0x7faa78039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.419 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.426+0000 7faa8affd640 1 -- 172.21.15.97:0/2058688572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7faa7c038040 con 0x2086ad0 2024-03-27T08:38:45.419 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.427+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> v2:172.21.15.97:6801/3203341735 conn(0x2119880 0x2105080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.419 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.427+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 --> v2:172.21.15.97:6801/3203341735 -- osd_op(unknown.0.0:1 200.5 200:a557efb1:::obj8:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x2107dc0 con 0x2119880 2024-03-27T08:38:45.419 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.427+0000 7faa9199a640 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7faa78037030 0x7faa78039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.420 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.427+0000 7faa9199a640 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7faa78037030 0x7faa78039540 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.420 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.427+0000 7faa9299c640 1 --2- 172.21.15.97:0/2058688572 >> v2:172.21.15.97:6801/3203341735 conn(0x2119880 0x2105080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:45.422 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.431+0000 7faa9299c640 1 --2- 172.21.15.97:0/2058688572 >> v2:172.21.15.97:6801/3203341735 conn(0x2119880 0x2105080 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.434 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9299c640 1 -- 172.21.15.97:0/2058688572 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== osd_op_reply(1 obj8 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7faa8400c430 con 0x2119880 2024-03-27T08:38:45.434 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 >> v2:172.21.15.97:6801/3203341735 conn(0x2119880 msgr2=0x2105080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> v2:172.21.15.97:6801/3203341735 conn(0x2119880 0x2105080 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7faa78037030 msgr2=0x7faa78039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7faa78037030 0x7faa78039540 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 msgr2=0x2118400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x2118400 crc :-1 s=READY pgs=12898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 shutdown_connections 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> v2:172.21.15.97:6801/3203341735 conn(0x2119880 0x2105080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7faa78037030 0x7faa78039540 unknown :-1 s=CLOSED pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.441+0000 7faa9a036ec0 1 --2- 172.21.15.97:0/2058688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x2086ad0 0x2118400 unknown :-1 s=CLOSED pgs=12898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.442+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 >> 172.21.15.97:0/2058688572 conn(0x1fa0ea0 msgr2=0x21a26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.442+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 shutdown_connections 2024-03-27T08:38:45.435 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.442+0000 7faa9a036ec0 1 -- 172.21.15.97:0/2058688572 wait complete. 2024-03-27T08:38:45.436 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:45.436 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj9 /etc/passwd 2024-03-27T08:38:45.470 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.479+0000 7f882aa59ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.475 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.484+0000 7f882aa59ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.475 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.484+0000 7f882aa59ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.476 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.484+0000 7f882aa59ec0 1 Processor -- start 2024-03-27T08:38:45.476 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.484+0000 7f882aa59ec0 1 -- start start 2024-03-27T08:38:45.478 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.485+0000 7f882aa59ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xcf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.478 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.485+0000 7f882aa59ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xbcb0b0 con 0xcf7360 2024-03-27T08:38:45.478 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.485+0000 7f8822b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xcf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.478 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.485+0000 7f8822b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xcf4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51080/0 (socket says 172.21.15.97:51080) 2024-03-27T08:38:45.478 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.485+0000 7f8822b9b640 1 -- 172.21.15.97:0/569061795 learned_addr learned my addr 172.21.15.97:0/569061795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.478 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f8822b9b640 1 -- 172.21.15.97:0/569061795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb6b1e0 con 0xcf7360 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f8822b9b640 1 --2- 172.21.15.97:0/569061795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xcf4320 crc :-1 s=READY pgs=12901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58eefd4300afe030 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f8821398640 1 -- 172.21.15.97:0/569061795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f880c00b5d0 con 0xcf7360 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f8821398640 1 -- 172.21.15.97:0/569061795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f880c00b730 con 0xcf7360 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f8821398640 1 -- 172.21.15.97:0/569061795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f880c00b9d0 con 0xcf7360 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f882aa59ec0 1 -- 172.21.15.97:0/569061795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 msgr2=0xcf4320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.486+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/569061795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xcf4320 crc :-1 s=READY pgs=12901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 -- 172.21.15.97:0/569061795 shutdown_connections 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/569061795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xcf4320 unknown :-1 s=CLOSED pgs=12901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 -- 172.21.15.97:0/569061795 >> 172.21.15.97:0/569061795 conn(0xb7fea0 msgr2=0xcf2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 -- 172.21.15.97:0/569061795 shutdown_connections 2024-03-27T08:38:45.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 -- 172.21.15.97:0/569061795 wait complete. 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 Processor -- start 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.487+0000 7f882aa59ec0 1 -- start start 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.488+0000 7f882aa59ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xc60060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.488+0000 7f882aa59ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xbc8b30 con 0xcf7360 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.488+0000 7f8822b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xc60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.488+0000 7f8822b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xc60060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51084/0 (socket says 172.21.15.97:51084) 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.488+0000 7f8822b9b640 1 -- 172.21.15.97:0/915315547 learned_addr learned my addr 172.21.15.97:0/915315547 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f8822b9b640 1 -- 172.21.15.97:0/915315547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xb5de30 con 0xcf7360 2024-03-27T08:38:45.481 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f8822b9b640 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xc60060 crc :-1 s=READY pgs=12902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.482 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f881b7fe640 1 -- 172.21.15.97:0/915315547 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f880c012940 con 0xcf7360 2024-03-27T08:38:45.482 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f881b7fe640 1 -- 172.21.15.97:0/915315547 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f880c005040 con 0xcf7360 2024-03-27T08:38:45.482 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xb5d890 con 0xcf7360 2024-03-27T08:38:45.482 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f881b7fe640 1 -- 172.21.15.97:0/915315547 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f880c012bd0 con 0xcf7360 2024-03-27T08:38:45.482 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.489+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0xc5fca0 con 0xcf7360 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.490+0000 7f881b7fe640 1 -- 172.21.15.97:0/915315547 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f880c00d040 con 0xcf7360 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.490+0000 7f881b7fe640 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8808037030 0x7f8808039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.490+0000 7f881b7fe640 1 -- 172.21.15.97:0/915315547 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7f880c00a040 con 0xcf7360 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.490+0000 7f882239a640 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8808037030 0x7f8808039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.490+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> v2:172.21.15.190:6805/971408905 conn(0xcfa190 0xd8bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.490+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 200.7 200:fe351e27:::obj9:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0xd80da0 con 0xcfa190 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.491+0000 7f882239a640 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8808037030 0x7f8808039540 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.483 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.491+0000 7f882339c640 1 --2- 172.21.15.97:0/915315547 >> v2:172.21.15.190:6805/971408905 conn(0xcfa190 0xd8bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:45.486 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.495+0000 7f882339c640 1 --2- 172.21.15.97:0/915315547 >> v2:172.21.15.190:6805/971408905 conn(0xcfa190 0xd8bc50 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.498 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882339c640 1 -- 172.21.15.97:0/915315547 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj9 [writefull 0~11812] v688'2 uv2 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f881400a430 con 0xcfa190 2024-03-27T08:38:45.498 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 >> v2:172.21.15.190:6805/971408905 conn(0xcfa190 msgr2=0xd8bc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.498 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> v2:172.21.15.190:6805/971408905 conn(0xcfa190 0xd8bc50 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.498 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8808037030 msgr2=0x7f8808039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.498 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8808037030 0x7f8808039540 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 msgr2=0xc60060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xc60060 crc :-1 s=READY pgs=12902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 shutdown_connections 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> v2:172.21.15.190:6805/971408905 conn(0xcfa190 0xd8bc50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8808037030 0x7f8808039540 unknown :-1 s=CLOSED pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 --2- 172.21.15.97:0/915315547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xcf7360 0xc60060 unknown :-1 s=CLOSED pgs=12902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 >> 172.21.15.97:0/915315547 conn(0xb7fea0 msgr2=0xcf2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 shutdown_connections 2024-03-27T08:38:45.499 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.506+0000 7f882aa59ec0 1 -- 172.21.15.97:0/915315547 wait complete. 2024-03-27T08:38:45.500 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:38:45.500 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put obj10 /etc/passwd 2024-03-27T08:38:45.535 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.543+0000 7fdc47d22ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.540 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.548+0000 7fdc47d22ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.540 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.548+0000 7fdc47d22ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:38:45.540 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.549+0000 7fdc47d22ec0 1 Processor -- start 2024-03-27T08:38:45.541 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.549+0000 7fdc47d22ec0 1 -- start start 2024-03-27T08:38:45.542 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.550+0000 7fdc47d22ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x266f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.542 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.550+0000 7fdc47d22ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x25430b0 con 0x25d70a0 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.550+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x266f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.550+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x266f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51110/0 (socket says 172.21.15.97:51110) 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.550+0000 7fdc3b7fe640 1 -- 172.21.15.97:0/1968371952 learned_addr learned my addr 172.21.15.97:0/1968371952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.550+0000 7fdc3b7fe640 1 -- 172.21.15.97:0/1968371952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x24e31e0 con 0x25d70a0 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc3b7fe640 1 --2- 172.21.15.97:0/1968371952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x266f9d0 crc :-1 s=READY pgs=12905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300c28969c497396 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc39ffb640 1 -- 172.21.15.97:0/1968371952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc2c02aeb0 con 0x25d70a0 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc39ffb640 1 -- 172.21.15.97:0/1968371952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdc2c0333d0 con 0x25d70a0 2024-03-27T08:38:45.543 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc39ffb640 1 -- 172.21.15.97:0/1968371952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc2c033670 con 0x25d70a0 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/1968371952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 msgr2=0x266f9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/1968371952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x266f9d0 crc :-1 s=READY pgs=12905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/1968371952 shutdown_connections 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/1968371952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x266f9d0 unknown :-1 s=CLOSED pgs=12905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/1968371952 >> 172.21.15.97:0/1968371952 conn(0x24f7ea0 msgr2=0x2670d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/1968371952 shutdown_connections 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.551+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/1968371952 wait complete. 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.552+0000 7fdc47d22ec0 1 Processor -- start 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.552+0000 7fdc47d22ec0 1 -- start start 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.553+0000 7fdc47d22ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x27086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.553+0000 7fdc47d22ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x2540b30 con 0x25d70a0 2024-03-27T08:38:45.546 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.553+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x27086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.553+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x27086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51114/0 (socket says 172.21.15.97:51114) 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.553+0000 7fdc3b7fe640 1 -- 172.21.15.97:0/2371864 learned_addr learned my addr 172.21.15.97:0/2371864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc3b7fe640 1 -- 172.21.15.97:0/2371864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x24d5e30 con 0x25d70a0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc3b7fe640 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x27086e0 crc :-1 s=READY pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc38ff9640 1 -- 172.21.15.97:0/2371864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc2c03a040 con 0x25d70a0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc38ff9640 1 -- 172.21.15.97:0/2371864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdc2c034650 con 0x25d70a0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc38ff9640 1 -- 172.21.15.97:0/2371864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdc2c0348f0 con 0x25d70a0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x24d5890 con 0x25d70a0 2024-03-27T08:38:45.547 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.554+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2701150 con 0x25d70a0 2024-03-27T08:38:45.548 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.555+0000 7fdc38ff9640 1 -- 172.21.15.97:0/2371864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fdc2c03d040 con 0x25d70a0 2024-03-27T08:38:45.548 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.555+0000 7fdc38ff9640 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdc1c037030 0x7fdc1c039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.548 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.555+0000 7fdc38ff9640 1 -- 172.21.15.97:0/2371864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(688..688 src has 399..688) ==== 3791+0+0 (crc 0 0 0) 0x7fdc2c02e040 con 0x25d70a0 2024-03-27T08:38:45.548 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.555+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> v2:172.21.15.97:6801/3203341735 conn(0x25de060 0x2658c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:38:45.549 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.555+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 --> v2:172.21.15.97:6801/3203341735 -- osd_op(unknown.0.0:1 200.b 200:d1337354:::obj10:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e688) -- 0x265ad10 con 0x25de060 2024-03-27T08:38:45.549 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.555+0000 7fdc3affd640 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdc1c037030 0x7fdc1c039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:38:45.549 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.556+0000 7fdc3affd640 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdc1c037030 0x7fdc1c039540 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.549 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.556+0000 7fdc4099c640 1 --2- 172.21.15.97:0/2371864 >> v2:172.21.15.97:6801/3203341735 conn(0x25de060 0x2658c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:38:45.551 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.559+0000 7fdc4099c640 1 --2- 172.21.15.97:0/2371864 >> v2:172.21.15.97:6801/3203341735 conn(0x25de060 0x2658c70 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:38:45.563 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc4099c640 1 -- 172.21.15.97:0/2371864 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== osd_op_reply(1 obj10 [writefull 0~11812] v688'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fdc3400c430 con 0x25de060 2024-03-27T08:38:45.563 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 >> v2:172.21.15.97:6801/3203341735 conn(0x25de060 msgr2=0x2658c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.563 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> v2:172.21.15.97:6801/3203341735 conn(0x25de060 0x2658c70 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.563 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdc1c037030 msgr2=0x7fdc1c039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.563 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdc1c037030 0x7fdc1c039540 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 msgr2=0x27086e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x27086e0 crc :-1 s=READY pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 shutdown_connections 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> v2:172.21.15.97:6801/3203341735 conn(0x25de060 0x2658c70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdc1c037030 0x7fdc1c039540 unknown :-1 s=CLOSED pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 --2- 172.21.15.97:0/2371864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x25d70a0 0x27086e0 unknown :-1 s=CLOSED pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 >> 172.21.15.97:0/2371864 conn(0x24f7ea0 msgr2=0x2670d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 shutdown_connections 2024-03-27T08:38:45.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:38:45.571+0000 7fdc47d22ec0 1 -- 172.21.15.97:0/2371864 wait complete. 2024-03-27T08:38:45.565 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-27T08:39:15.567 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=99398 2024-03-27T08:39:15.568 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_objects 100 2024-03-27T08:39:15.568 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put onemore /etc/passwd 2024-03-27T08:39:15.612 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.618+0000 7fd2f1649ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:15.615 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.624+0000 7fd2f1649ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:15.615 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.624+0000 7fd2f1649ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:15.616 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.624+0000 7fd2f1649ec0 1 Processor -- start 2024-03-27T08:39:15.616 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.625+0000 7fd2f1649ec0 1 -- start start 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.625+0000 7fd2f1649ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 0xf1a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2f1649ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xe7c0b0 con 0xfa8a80 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 0xf1a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 0xf1a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54202/0 (socket says 172.21.15.97:54202) 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e979b640 1 -- 172.21.15.97:0/1917228083 learned_addr learned my addr 172.21.15.97:0/1917228083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e979b640 1 -- 172.21.15.97:0/1917228083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xe1c1e0 con 0xfa8a80 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e979b640 1 --2- 172.21.15.97:0/1917228083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 0xf1a3a0 crc :-1 s=READY pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa00036145bfb142 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e2ffd640 1 -- 172.21.15.97:0/1917228083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd2d400b500 con 0xfa8a80 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e2ffd640 1 -- 172.21.15.97:0/1917228083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd2d400b660 con 0xfa8a80 2024-03-27T08:39:15.618 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.626+0000 7fd2e2ffd640 1 -- 172.21.15.97:0/1917228083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd2d400b900 con 0xfa8a80 2024-03-27T08:39:15.619 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/1917228083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 msgr2=0xf1a3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/1917228083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 0xf1a3a0 crc :-1 s=READY pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/1917228083 shutdown_connections 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/1917228083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfa8a80 0xf1a3a0 unknown :-1 s=CLOSED pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/1917228083 >> 172.21.15.97:0/1917228083 conn(0xe30ea0 msgr2=0xfadcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/1917228083 shutdown_connections 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.627+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/1917228083 wait complete. 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.628+0000 7fd2f1649ec0 1 Processor -- start 2024-03-27T08:39:15.620 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.628+0000 7fd2f1649ec0 1 -- start start 2024-03-27T08:39:15.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.629+0000 7fd2f1649ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 0xf94b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.629+0000 7fd2f1649ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xe79b30 con 0xfaedf0 2024-03-27T08:39:15.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.629+0000 7fd2e979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 0xf94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:15.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.629+0000 7fd2e979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 0xf94b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54216/0 (socket says 172.21.15.97:54216) 2024-03-27T08:39:15.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.629+0000 7fd2e979b640 1 -- 172.21.15.97:0/139419717 learned_addr learned my addr 172.21.15.97:0/139419717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:15.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2e979b640 1 -- 172.21.15.97:0/139419717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xe0ee30 con 0xfaedf0 2024-03-27T08:39:15.622 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2e979b640 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 0xf94b00 crc :-1 s=READY pgs=12910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.622 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd2d40110b0 con 0xfaedf0 2024-03-27T08:39:15.622 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd2d4011210 con 0xfaedf0 2024-03-27T08:39:15.623 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xe0e890 con 0xfaedf0 2024-03-27T08:39:15.623 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd2d4011a60 con 0xfaedf0 2024-03-27T08:39:15.623 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.630+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0xfa6eb0 con 0xfaedf0 2024-03-27T08:39:15.624 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.632+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fd2d4011ca0 con 0xfaedf0 2024-03-27T08:39:15.624 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.632+0000 7fd2e1ffb640 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fd2d0036fe0 0x7fd2d00394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.624 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.632+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(690..690 src has 399..690) ==== 3791+0+0 (crc 0 0 0) 0x7fd2d4048850 con 0xfaedf0 2024-03-27T08:39:15.624 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.632+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=691}) -- 0x7fd2d003d250 con 0xfaedf0 2024-03-27T08:39:15.624 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.632+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> v2:172.21.15.97:6801/3203341735 conn(0xfa5e10 0xfa6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.624 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.632+0000 7fd2e8f9a640 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fd2d0036fe0 0x7fd2d00394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:15.625 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.633+0000 7fd2e8f9a640 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fd2d0036fe0 0x7fd2d00394f0 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.625 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.633+0000 7fd2e9f9c640 1 --2- 172.21.15.97:0/139419717 >> v2:172.21.15.97:6801/3203341735 conn(0xfa5e10 0xfa6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:39:15.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.638+0000 7fd2e9f9c640 1 --2- 172.21.15.97:0/139419717 >> v2:172.21.15.97:6801/3203341735 conn(0xfa5e10 0xfa6220 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.672+0000 7f2edea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:15.672 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.681+0000 7f2edea6d640 1 Processor -- start 2024-03-27T08:39:15.672 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.681+0000 7f2edea6d640 1 -- start start 2024-03-27T08:39:15.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2edea6d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ed809f980 0x7f2ed809fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2edea6d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed809f2e0 con 0x7f2ed8134680 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2edea6d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed806ea70 con 0x7f2ed809f980 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2edc809640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ed809f980 0x7f2ed809fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2ed7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ed8134680 0x7f2ed8134a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43882/0 (socket says 172.21.15.97:43882) 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2edc809640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ed809f980 0x7f2ed809fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54222/0 (socket says 172.21.15.97:54222) 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.682+0000 7f2ed7fff640 1 -- 172.21.15.97:0/1189425391 learned_addr learned my addr 172.21.15.97:0/1189425391 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2edc809640 1 -- 172.21.15.97:0/1189425391 >> v1:172.21.15.97:6789/0 conn(0x7f2ed8134680 legacy=0x7f2ed8134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2edc809640 1 -- 172.21.15.97:0/1189425391 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80a0900 con 0x7f2ed809f980 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2ed77fe640 1 -- 172.21.15.97:0/1189425391 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2515681827 0 0) 0x7f2ed809f2e0 con 0x7f2ed8134680 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2edc809640 1 --2- 172.21.15.97:0/1189425391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed809fd90 crc :-1 s=READY pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f864522d264d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2ed77fe640 1 -- 172.21.15.97:0/1189425391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2ecc009d90 con 0x7f2ed809f980 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2ed77fe640 1 -- 172.21.15.97:0/1189425391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2ecc00e230 con 0x7f2ed809f980 2024-03-27T08:39:15.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.683+0000 7f2ed77fe640 1 -- 172.21.15.97:0/1189425391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2ecc00e4d0 con 0x7f2ed809f980 2024-03-27T08:39:15.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 -- 172.21.15.97:0/1189425391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 msgr2=0x7f2ed809fd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:15.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 --2- 172.21.15.97:0/1189425391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed809fd90 crc :-1 s=READY pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:15.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 -- 172.21.15.97:0/1189425391 shutdown_connections 2024-03-27T08:39:15.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 --2- 172.21.15.97:0/1189425391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed809fd90 unknown :-1 s=CLOSED pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:15.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 -- 172.21.15.97:0/1189425391 >> 172.21.15.97:0/1189425391 conn(0x7f2ed80a4bc0 msgr2=0x7f2ed80a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:15.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 -- 172.21.15.97:0/1189425391 shutdown_connections 2024-03-27T08:39:15.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 1 -- 172.21.15.97:0/1189425391 wait complete. 2024-03-27T08:39:15.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.684+0000 7f2edea6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:15.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.685+0000 7f2edea6d640 1 Processor -- start 2024-03-27T08:39:15.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.685+0000 7f2edea6d640 1 -- start start 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.686+0000 7f2edea6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed811e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.686+0000 7f2edea6d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81206f0 con 0x7f2ed809f980 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edc809640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed811e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edc809640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed811e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54226/0 (socket says 172.21.15.97:54226) 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edc809640 1 -- 172.21.15.97:0/2021363802 learned_addr learned my addr 172.21.15.97:0/2021363802 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edc809640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed8120c60 con 0x7f2ed809f980 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edc809640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed811e110 crc :-1 s=READY pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2ecc014620 con 0x7f2ed809f980 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2ecc014780 con 0x7f2ed809f980 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed8120e90 con 0x7f2ed809f980 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2ecc014a20 con 0x7f2ed809f980 2024-03-27T08:39:15.679 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.687+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81221a0 con 0x7f2ed809f980 2024-03-27T08:39:15.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.688+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2ecc014bd0 con 0x7f2ed809f980 2024-03-27T08:39:15.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.688+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed80693c0 con 0x7f2ed809f980 2024-03-27T08:39:15.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.688+0000 7f2ed57fa640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2ebc0387d0 0x7f2ebc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:15.681 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.688+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(690..690 src has 399..690) ==== 3791+0+0 (crc 0 0 0) 0x7f2ecc019040 con 0x7f2ed809f980 2024-03-27T08:39:15.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.689+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=691}) -- 0x7f2ebc03e9e0 con 0x7f2ed809f980 2024-03-27T08:39:15.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.689+0000 7f2ed7fff640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2ebc0387d0 0x7f2ebc03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:15.682 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.689+0000 7f2ed7fff640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2ebc0387d0 0x7f2ebc03ace0 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:15.685 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.694+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f2ecc011380 con 0x7f2ed809f980 2024-03-27T08:39:15.861 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:15.869+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "100"} v 0) -- 0x7f2ed80735d0 con 0x7f2ed809f980 2024-03-27T08:39:16.847 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:16.856+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v691) ==== 225+0+0 (crc 0 0 0) 0x7f2ecc010310 con 0x7f2ed809f980 2024-03-27T08:39:16.849 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:16.858+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 6 ==== osd_map(691..691 src has 399..691) ==== 628+0+0 (crc 0 0 0) 0x7fd2d40084a0 con 0xfaedf0 2024-03-27T08:39:16.850 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:16.858+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(691..691 src has 399..691) ==== 628+0+0 (crc 0 0 0) 0x7f2ecc00fcd0 con 0x7f2ed809f980 2024-03-27T08:39:16.850 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:16.858+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=692}) -- 0x7f2ebc03f920 con 0x7f2ed809f980 2024-03-27T08:39:16.850 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:16.858+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=692}) -- 0x7fd2d003d8e0 con 0xfaedf0 2024-03-27T08:39:16.957 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:16.965+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "100"} v 0) -- 0x7f2ed81209f0 con 0x7f2ed809f980 2024-03-27T08:39:17.820 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.828+0000 7f2ed57fa640 1 -- 172.21.15.97:0/2021363802 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v692) ==== 225+0+0 (crc 0 0 0) 0x7f2ecc00f2a0 con 0x7f2ed809f980 2024-03-27T08:39:17.820 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 100 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:39:17.822 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2ebc0387d0 msgr2=0x7f2ebc03ace0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:17.822 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2ebc0387d0 0x7f2ebc03ace0 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:17.822 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 msgr2=0x7f2ed811e110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed811e110 crc :-1 s=READY pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 shutdown_connections 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2ebc0387d0 0x7f2ebc03ace0 unknown :-1 s=CLOSED pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 --2- 172.21.15.97:0/2021363802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed809f980 0x7f2ed811e110 unknown :-1 s=CLOSED pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 >> 172.21.15.97:0/2021363802 conn(0x7f2ed80a4bc0 msgr2=0x7f2ed8133130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 shutdown_connections 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.830+0000 7f2edea6d640 1 -- 172.21.15.97:0/2021363802 wait complete. 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.831+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 <== mon.0 v2:172.21.15.97:3300/0 7 ==== osd_map(692..692 src has 399..692) ==== 628+0+0 (crc 0 0 0) 0x7fd2d40062a0 con 0xfaedf0 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.831+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=693}) -- 0x7fd2d003db00 con 0xfaedf0 2024-03-27T08:39:17.823 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:17.831+0000 7fd2e1ffb640 1 -- 172.21.15.97:0/139419717 --> v2:172.21.15.97:6801/3203341735 -- osd_op(unknown.0.0:1 200.5 200:b9ff7440:::onemore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e692) -- 0x7fd2d003dcf0 con 0xfa5e10 2024-03-27T08:39:17.837 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 99398 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2e9f9c640 1 -- 172.21.15.97:0/139419717 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== osd_op_reply(1 onemore [writefull 0~11812] v692'2 uv2 ondisk = 0) ==== 151+0+0 (crc 0 0 0) 0x7fd2dc00a430 con 0xfa5e10 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 >> v2:172.21.15.97:6801/3203341735 conn(0xfa5e10 msgr2=0xfa6220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> v2:172.21.15.97:6801/3203341735 conn(0xfa5e10 0xfa6220 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fd2d0036fe0 msgr2=0x7fd2d00394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fd2d0036fe0 0x7fd2d00394f0 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 msgr2=0xf94b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.028+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 0xf94b00 crc :-1 s=READY pgs=12910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 shutdown_connections 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> v2:172.21.15.97:6801/3203341735 conn(0xfa5e10 0xfa6220 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fd2d0036fe0 0x7fd2d00394f0 unknown :-1 s=CLOSED pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 --2- 172.21.15.97:0/139419717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xfaedf0 0xf94b00 unknown :-1 s=CLOSED pgs=12910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 >> 172.21.15.97:0/139419717 conn(0xe30ea0 msgr2=0xfadcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 shutdown_connections 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.029+0000 7fd2f1649ec0 1 -- 172.21.15.97:0/139419717 wait complete. 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -ne 0 ']' 2024-03-27T08:39:18.022 INFO:tasks.workunit.client.0.smithi097.stderr:+ true 2024-03-27T08:39:18.023 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put twomore /etc/passwd 2024-03-27T08:39:18.057 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.065+0000 7ff517637ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:18.061 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.070+0000 7ff517637ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:18.061 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.070+0000 7ff517637ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:18.062 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.071+0000 7ff517637ec0 1 Processor -- start 2024-03-27T08:39:18.062 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.071+0000 7ff517637ec0 1 -- start start 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff517637ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x112b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff517637ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x10950b0 con 0x112fb20 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50f79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x112b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50f79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x112b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54238/0 (socket says 172.21.15.97:54238) 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50f79b640 1 -- 172.21.15.97:0/3254827675 learned_addr learned my addr 172.21.15.97:0/3254827675 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50f79b640 1 -- 172.21.15.97:0/3254827675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x10351e0 con 0x112fb20 2024-03-27T08:39:18.064 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50f79b640 1 --2- 172.21.15.97:0/3254827675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x112b680 crc :-1 s=READY pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ff9382105d8656 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.065 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50df98640 1 -- 172.21.15.97:0/3254827675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff4f800c490 con 0x112fb20 2024-03-27T08:39:18.065 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50df98640 1 -- 172.21.15.97:0/3254827675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff4f800c5f0 con 0x112fb20 2024-03-27T08:39:18.065 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.072+0000 7ff50df98640 1 -- 172.21.15.97:0/3254827675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff4f800c890 con 0x112fb20 2024-03-27T08:39:18.066 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 -- 172.21.15.97:0/3254827675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 msgr2=0x112b680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.066 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3254827675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x112b680 crc :-1 s=READY pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.066 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 -- 172.21.15.97:0/3254827675 shutdown_connections 2024-03-27T08:39:18.066 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3254827675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x112b680 unknown :-1 s=CLOSED pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 -- 172.21.15.97:0/3254827675 >> 172.21.15.97:0/3254827675 conn(0x1049ea0 msgr2=0x1129640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:18.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 -- 172.21.15.97:0/3254827675 shutdown_connections 2024-03-27T08:39:18.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.073+0000 7ff517637ec0 1 -- 172.21.15.97:0/3254827675 wait complete. 2024-03-27T08:39:18.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.074+0000 7ff517637ec0 1 Processor -- start 2024-03-27T08:39:18.071 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.074+0000 7ff517637ec0 1 -- start start 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff517637ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x125a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff517637ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1092b30 con 0x112fb20 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff50f79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x125a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff50f79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x125a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54248/0 (socket says 172.21.15.97:54248) 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff50f79b640 1 -- 172.21.15.97:0/3588263079 learned_addr learned my addr 172.21.15.97:0/3588263079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff50f79b640 1 -- 172.21.15.97:0/3588263079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1027e30 con 0x112fb20 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff50f79b640 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x125a0b0 crc :-1 s=READY pgs=12917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.075+0000 7ff50cf96640 1 -- 172.21.15.97:0/3588263079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff4f8016040 con 0x112fb20 2024-03-27T08:39:18.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.076+0000 7ff50cf96640 1 -- 172.21.15.97:0/3588263079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff4f8011b00 con 0x112fb20 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.076+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1027890 con 0x112fb20 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.076+0000 7ff50cf96640 1 -- 172.21.15.97:0/3588263079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff4f8011da0 con 0x112fb20 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.076+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x11aea00 con 0x112fb20 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.076+0000 7ff50cf96640 1 -- 172.21.15.97:0/3588263079 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ff4f800e440 con 0x112fb20 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.077+0000 7ff50cf96640 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff4f4037030 0x7ff4f4039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.077+0000 7ff50cf96640 1 -- 172.21.15.97:0/3588263079 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(692..692 src has 399..692) ==== 3791+0+0 (crc 0 0 0) 0x7ff4f8048a80 con 0x112fb20 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.077+0000 7ff50ef9a640 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff4f4037030 0x7ff4f4039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.077+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> v2:172.21.15.97:6805/1668842645 conn(0x1130640 0x11ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.077+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 200.6 200:7019d845:::twomore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e692) -- 0x11b0c10 con 0x1130640 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.077+0000 7ff50ef9a640 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff4f4037030 0x7ff4f4039540 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.078+0000 7ff50ff9c640 1 --2- 172.21.15.97:0/3588263079 >> v2:172.21.15.97:6805/1668842645 conn(0x1130640 0x11ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:39:18.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.081+0000 7ff50ff9c640 1 --2- 172.21.15.97:0/3588263079 >> v2:172.21.15.97:6805/1668842645 conn(0x1130640 0x11ad870 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.254+0000 7ff50ff9c640 1 -- 172.21.15.97:0/3588263079 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 twomore [writefull 0~11812] v692'1 uv1 ondisk = 0) ==== 151+0+0 (crc 0 0 0) 0x7ff50000a430 con 0x1130640 2024-03-27T08:39:18.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 >> v2:172.21.15.97:6805/1668842645 conn(0x1130640 msgr2=0x11ad870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> v2:172.21.15.97:6805/1668842645 conn(0x1130640 0x11ad870 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.247 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff4f4037030 msgr2=0x7ff4f4039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff4f4037030 0x7ff4f4039540 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 msgr2=0x125a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x125a0b0 crc :-1 s=READY pgs=12917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 shutdown_connections 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> v2:172.21.15.97:6805/1668842645 conn(0x1130640 0x11ad870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ff4f4037030 0x7ff4f4039540 unknown :-1 s=CLOSED pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 --2- 172.21.15.97:0/3588263079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x112fb20 0x125a0b0 unknown :-1 s=CLOSED pgs=12917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 >> 172.21.15.97:0/3588263079 conn(0x1049ea0 msgr2=0x1129640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 shutdown_connections 2024-03-27T08:39:18.248 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.255+0000 7ff517637ec0 1 -- 172.21.15.97:0/3588263079 wait complete. 2024-03-27T08:39:18.249 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_bytes 100 2024-03-27T08:39:18.341 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.341+0000 7fc8b015d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:18.342 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.350+0000 7fc8b015d640 1 Processor -- start 2024-03-27T08:39:18.342 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.350+0000 7fc8b015d640 1 -- start start 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8b015d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8a812e500 0x7fc8a812e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8b015d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8a812e2d0 con 0x7fc8a812ee50 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8b015d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a812f260 con 0x7fc8a812e500 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8adef9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8a812e500 0x7fc8a812e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8adef9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8a812e500 0x7fc8a812e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54262/0 (socket says 172.21.15.97:54262) 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8adef9640 1 -- 172.21.15.97:0/4128765826 learned_addr learned my addr 172.21.15.97:0/4128765826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.351+0000 7fc8ad6f8640 1 --1- 172.21.15.97:0/4128765826 >> v1:172.21.15.97:6789/0 conn(0x7fc8a812ee50 0x7fc8a809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43896/0 (socket says 172.21.15.97:43896) 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8adef9640 1 -- 172.21.15.97:0/4128765826 >> v1:172.21.15.97:6789/0 conn(0x7fc8a812ee50 legacy=0x7fc8a809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8acef7640 1 -- 172.21.15.97:0/4128765826 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990620658 0 0) 0x7fc8a812e2d0 con 0x7fc8a812ee50 2024-03-27T08:39:18.344 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8adef9640 1 -- 172.21.15.97:0/4128765826 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a809a170 con 0x7fc8a812e500 2024-03-27T08:39:18.345 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8adef9640 1 --2- 172.21.15.97:0/4128765826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a812e910 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab13ae2d05d41cae server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.345 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8acef7640 1 -- 172.21.15.97:0/4128765826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc8a000e040 con 0x7fc8a812e500 2024-03-27T08:39:18.345 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8acef7640 1 -- 172.21.15.97:0/4128765826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc8a00091e0 con 0x7fc8a812e500 2024-03-27T08:39:18.345 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.352+0000 7fc8acef7640 1 -- 172.21.15.97:0/4128765826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc8a0007730 con 0x7fc8a812e500 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 -- 172.21.15.97:0/4128765826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 msgr2=0x7fc8a812e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 --2- 172.21.15.97:0/4128765826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a812e910 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 -- 172.21.15.97:0/4128765826 shutdown_connections 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 --2- 172.21.15.97:0/4128765826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a812e910 unknown :-1 s=CLOSED pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 -- 172.21.15.97:0/4128765826 >> 172.21.15.97:0/4128765826 conn(0x7fc8a80a0b00 msgr2=0x7fc8a80a0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 -- 172.21.15.97:0/4128765826 shutdown_connections 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 1 -- 172.21.15.97:0/4128765826 wait complete. 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.353+0000 7fc8b015d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:18.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.354+0000 7fc8b015d640 1 Processor -- start 2024-03-27T08:39:18.347 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.354+0000 7fc8b015d640 1 -- start start 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.355+0000 7fc8b015d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a8071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.355+0000 7fc8b015d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a80731a0 con 0x7fc8a812e500 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8adef9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a8071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8adef9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a8071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54278/0 (socket says 172.21.15.97:54278) 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8adef9640 1 -- 172.21.15.97:0/2432857051 learned_addr learned my addr 172.21.15.97:0/2432857051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8adef9640 1 -- 172.21.15.97:0/2432857051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a812b120 con 0x7fc8a812e500 2024-03-27T08:39:18.348 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8adef9640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a8071620 crc :-1 s=READY pgs=12920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.349 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc8a000e040 con 0x7fc8a812e500 2024-03-27T08:39:18.349 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc8a00094d0 con 0x7fc8a812e500 2024-03-27T08:39:18.349 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a806c440 con 0x7fc8a812e500 2024-03-27T08:39:18.349 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc8a0011920 con 0x7fc8a812e500 2024-03-27T08:39:18.349 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.356+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a812ada0 con 0x7fc8a812e500 2024-03-27T08:39:18.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.357+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a812a640 con 0x7fc8a812e500 2024-03-27T08:39:18.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.357+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fc8a0011ad0 con 0x7fc8a812e500 2024-03-27T08:39:18.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.357+0000 7fc8a67fc640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc8940387a0 0x7fc89403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:18.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.357+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(692..692 src has 399..692) ==== 3791+0+0 (crc 0 0 0) 0x7fc8a0049980 con 0x7fc8a812e500 2024-03-27T08:39:18.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.358+0000 7fc8ad6f8640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc8940387a0 0x7fc89403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:18.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.358+0000 7fc8ad6f8640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc8940387a0 0x7fc89403acb0 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:18.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.363+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fc8a812a640 con 0x7fc8a812e500 2024-03-27T08:39:18.530 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.539+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "100"} v 0) -- 0x7fc8a812a640 con 0x7fc8a812e500 2024-03-27T08:39:18.859 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.868+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v693) ==== 221+0+0 (crc 0 0 0) 0x7fc8a812a640 con 0x7fc8a812e500 2024-03-27T08:39:18.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:18.976+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "100"} v 0) -- 0x7fc8a812a640 con 0x7fc8a812e500 2024-03-27T08:39:19.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.874+0000 7fc8a67fc640 1 -- 172.21.15.97:0/2432857051 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v694) ==== 221+0+0 (crc 0 0 0) 0x7fc8a812a640 con 0x7fc8a812e500 2024-03-27T08:39:19.866 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_bytes = 100 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc8940387a0 msgr2=0x7fc89403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc8940387a0 0x7fc89403acb0 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 msgr2=0x7fc8a8071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a8071620 crc :-1 s=READY pgs=12920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 shutdown_connections 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc8940387a0 0x7fc89403acb0 unknown :-1 s=CLOSED pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 --2- 172.21.15.97:0/2432857051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a812e500 0x7fc8a8071620 unknown :-1 s=CLOSED pgs=12920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 >> 172.21.15.97:0/2432857051 conn(0x7fc8a80a0b00 msgr2=0x7fc8a81b9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:19.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 shutdown_connections 2024-03-27T08:39:19.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:19.876+0000 7fc8b015d640 1 -- 172.21.15.97:0/2432857051 wait complete. 2024-03-27T08:39:19.877 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-27T08:39:49.880 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=99489 2024-03-27T08:39:49.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_bytes 0 2024-03-27T08:39:49.881 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put two /etc/passwd 2024-03-27T08:39:49.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.928+0000 7fc1e1034ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:49.926 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.933+0000 7fc1e1034ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:49.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.934+0000 7fc1e1034ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:49.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.934+0000 7fc1e1034ec0 1 Processor -- start 2024-03-27T08:39:49.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.934+0000 7fc1e1034ec0 1 -- start start 2024-03-27T08:39:49.928 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.935+0000 7fc1e1034ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.928 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.935+0000 7fc1e1034ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x12b10b0 con 0x13450a0 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.935+0000 7fc1d899a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.935+0000 7fc1d899a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33408/0 (socket says 172.21.15.97:33408) 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.935+0000 7fc1d899a640 1 -- 172.21.15.97:0/182943901 learned_addr learned my addr 172.21.15.97:0/182943901 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.936+0000 7fc1d899a640 1 -- 172.21.15.97:0/182943901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x12511e0 con 0x13450a0 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.936+0000 7fc1d899a640 1 --2- 172.21.15.97:0/182943901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13c6760 crc :-1 s=READY pgs=12923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3177fc4afc3b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.936+0000 7fc1d2ffd640 1 -- 172.21.15.97:0/182943901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc1c000c490 con 0x13450a0 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.936+0000 7fc1d2ffd640 1 -- 172.21.15.97:0/182943901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc1c000c5f0 con 0x13450a0 2024-03-27T08:39:49.929 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.936+0000 7fc1d2ffd640 1 -- 172.21.15.97:0/182943901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc1c000c890 con 0x13450a0 2024-03-27T08:39:49.930 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.937+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/182943901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 msgr2=0x13c6760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:49.930 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.937+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/182943901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13c6760 crc :-1 s=READY pgs=12923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:49.930 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.937+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/182943901 shutdown_connections 2024-03-27T08:39:49.930 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.937+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/182943901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13c6760 unknown :-1 s=CLOSED pgs=12923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:49.931 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.937+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/182943901 >> 172.21.15.97:0/182943901 conn(0x1265ea0 msgr2=0x1347680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:49.931 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.937+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/182943901 shutdown_connections 2024-03-27T08:39:49.931 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.938+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/182943901 wait complete. 2024-03-27T08:39:49.931 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.939+0000 7fc1e1034ec0 1 Processor -- start 2024-03-27T08:39:49.931 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.939+0000 7fc1e1034ec0 1 -- start start 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.941+0000 7fc1e1034ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.941+0000 7fc1e1034ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x12aeb30 con 0x13450a0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.941+0000 7fc1d899a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.941+0000 7fc1d899a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13ddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33416/0 (socket says 172.21.15.97:33416) 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.941+0000 7fc1d899a640 1 -- 172.21.15.97:0/3656451783 learned_addr learned my addr 172.21.15.97:0/3656451783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1d899a640 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1243e30 con 0x13450a0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1d899a640 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13ddb10 crc :-1 s=READY pgs=12924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc1c00111e0 con 0x13450a0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc1c0011880 con 0x13450a0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc1c00119f0 con 0x13450a0 2024-03-27T08:39:49.936 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1243890 con 0x13450a0 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.942+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x13cc5a0 con 0x13450a0 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.943+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fc1c0011bf0 con 0x13450a0 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.943+0000 7fc1d1ffb640 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc1b8036fe0 0x7fc1b80394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.943+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(696..696 src has 399..696) ==== 3791+0+0 (crc 0 0 0) 0x7fc1c00488d0 con 0x13450a0 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.944+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=697}) -- 0x7fc1b803d1f0 con 0x13450a0 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.944+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> v2:172.21.15.97:6805/1668842645 conn(0x134c230 0x1476920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.937 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.944+0000 7fc1d3fff640 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc1b8036fe0 0x7fc1b80394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:49.938 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.944+0000 7fc1d3fff640 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc1b8036fe0 0x7fc1b80394f0 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.938 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.945+0000 7fc1d919b640 1 --2- 172.21.15.97:0/3656451783 >> v2:172.21.15.97:6805/1668842645 conn(0x134c230 0x1476920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:39:49.940 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.949+0000 7fc1d919b640 1 --2- 172.21.15.97:0/3656451783 >> v2:172.21.15.97:6805/1668842645 conn(0x134c230 0x1476920 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.980 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.980+0000 7f91feead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:49.982 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.989+0000 7f91feead640 1 Processor -- start 2024-03-27T08:39:49.982 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.990+0000 7f91feead640 1 -- start start 2024-03-27T08:39:49.983 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91feead640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91f809f860 0x7f91f809fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91feead640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f806d980 con 0x7f91f8134770 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91feead640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f91f806dc20 con 0x7f91f809f860 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91fcc49640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91f809f860 0x7f91f809fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91f7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f91f8134770 0x7f91f8134b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48116/0 (socket says 172.21.15.97:48116) 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91fcc49640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91f809f860 0x7f91f809fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33426/0 (socket says 172.21.15.97:33426) 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91f7fff640 1 -- 172.21.15.97:0/3079536275 learned_addr learned my addr 172.21.15.97:0/3079536275 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91fcc49640 1 -- 172.21.15.97:0/3079536275 >> v1:172.21.15.97:6789/0 conn(0x7f91f8134770 legacy=0x7f91f8134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91fcc49640 1 -- 172.21.15.97:0/3079536275 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f809f430 con 0x7f91f809f860 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91fcc49640 1 --2- 172.21.15.97:0/3079536275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f809fc70 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cea2af48fdf20d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91f77fe640 1 -- 172.21.15.97:0/3079536275 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f91ec009d90 con 0x7f91f809f860 2024-03-27T08:39:49.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91f77fe640 1 -- 172.21.15.97:0/3079536275 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f91ec00e230 con 0x7f91f809f860 2024-03-27T08:39:49.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.991+0000 7f91f77fe640 1 -- 172.21.15.97:0/3079536275 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f91ec00e4d0 con 0x7f91f809f860 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 -- 172.21.15.97:0/3079536275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 msgr2=0x7f91f809fc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 --2- 172.21.15.97:0/3079536275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f809fc70 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 -- 172.21.15.97:0/3079536275 shutdown_connections 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 --2- 172.21.15.97:0/3079536275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f809fc70 unknown :-1 s=CLOSED pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 -- 172.21.15.97:0/3079536275 >> 172.21.15.97:0/3079536275 conn(0x7f91f80a4bc0 msgr2=0x7f91f80a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 -- 172.21.15.97:0/3079536275 shutdown_connections 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 1 -- 172.21.15.97:0/3079536275 wait complete. 2024-03-27T08:39:49.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.992+0000 7f91feead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:49.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.994+0000 7f91feead640 1 Processor -- start 2024-03-27T08:39:49.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.994+0000 7f91feead640 1 -- start start 2024-03-27T08:39:49.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91feead640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f811e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91feead640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81205e0 con 0x7f91f809f860 2024-03-27T08:39:49.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91fcc49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f811e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91fcc49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f811e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33428/0 (socket says 172.21.15.97:33428) 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91fcc49640 1 -- 172.21.15.97:0/2658208682 learned_addr learned my addr 172.21.15.97:0/2658208682 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91fcc49640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f8120b70 con 0x7f91f809f860 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91fcc49640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f811e360 crc :-1 s=READY pgs=12927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.995+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f91ec014750 con 0x7f91f809f860 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.996+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f91ec0148b0 con 0x7f91f809f860 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.996+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f91ec014b50 con 0x7f91f809f860 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.996+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f8121c30 con 0x7f91f809f860 2024-03-27T08:39:49.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.996+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f81220b0 con 0x7f91f809f860 2024-03-27T08:39:49.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.996+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f80693c0 con 0x7f91f809f860 2024-03-27T08:39:49.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.996+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f91ec014d00 con 0x7f91f809f860 2024-03-27T08:39:49.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.997+0000 7f91f57fa640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f91dc0387a0 0x7f91dc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:49.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.997+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(696..696 src has 399..696) ==== 3791+0+0 (crc 0 0 0) 0x7f91ec019040 con 0x7f91f809f860 2024-03-27T08:39:49.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.997+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=697}) -- 0x7f91dc03e9b0 con 0x7f91f809f860 2024-03-27T08:39:49.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.997+0000 7f91f7fff640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f91dc0387a0 0x7f91dc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:49.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:49.998+0000 7f91f7fff640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f91dc0387a0 0x7f91dc03acb0 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:49.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.002+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f91ec011380 con 0x7f91f809f860 2024-03-27T08:39:50.169 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.177+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"} v 0) -- 0x7f91f8068d40 con 0x7f91f809f860 2024-03-27T08:39:50.849 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.857+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v697) ==== 217+0+0 (crc 0 0 0) 0x7f91ec010310 con 0x7f91f809f860 2024-03-27T08:39:50.849 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.857+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(697..697 src has 399..697) ==== 628+0+0 (crc 0 0 0) 0x7f91ec013e40 con 0x7f91f809f860 2024-03-27T08:39:50.849 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.857+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== osd_map(697..697 src has 399..697) ==== 628+0+0 (crc 0 0 0) 0x7fc1c0008680 con 0x13450a0 2024-03-27T08:39:50.849 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.857+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=698}) -- 0x7f91dc03fa10 con 0x7f91f809f860 2024-03-27T08:39:50.849 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.858+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=698}) -- 0x7fc1b803d880 con 0x13450a0 2024-03-27T08:39:50.956 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:50.965+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"} v 0) -- 0x7f91f8120880 con 0x7f91f809f860 2024-03-27T08:39:51.850 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.858+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v698) ==== 217+0+0 (crc 0 0 0) 0x7f91ec00f2a0 con 0x7f91f809f860 2024-03-27T08:39:51.850 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_bytes = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:39:51.850 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.858+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(698..698 src has 399..698) ==== 628+0+0 (crc 0 0 0) 0x7f91ec00fcd0 con 0x7f91f809f860 2024-03-27T08:39:51.851 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.858+0000 7f91f57fa640 1 -- 172.21.15.97:0/2658208682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=699}) -- 0x7f91dc03fac0 con 0x7f91f809f860 2024-03-27T08:39:51.851 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.858+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 7 ==== osd_map(698..698 src has 399..698) ==== 628+0+0 (crc 0 0 0) 0x7fc1c0009c30 con 0x13450a0 2024-03-27T08:39:51.851 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.858+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=699}) -- 0x7fc1b803daa0 con 0x13450a0 2024-03-27T08:39:51.853 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f91dc0387a0 msgr2=0x7f91dc03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:51.853 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f91dc0387a0 0x7f91dc03acb0 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:51.853 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 msgr2=0x7f91f811e360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:51.853 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f811e360 crc :-1 s=READY pgs=12927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:51.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 shutdown_connections 2024-03-27T08:39:51.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f91dc0387a0 0x7f91dc03acb0 unknown :-1 s=CLOSED pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:51.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 --2- 172.21.15.97:0/2658208682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f809f860 0x7f91f811e360 unknown :-1 s=CLOSED pgs=12927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:51.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 >> 172.21.15.97:0/2658208682 conn(0x7f91f80a4bc0 msgr2=0x7f91f8133000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:51.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 shutdown_connections 2024-03-27T08:39:51.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.861+0000 7f91feead640 1 -- 172.21.15.97:0/2658208682 wait complete. 2024-03-27T08:39:51.865 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_objects 0 2024-03-27T08:39:51.964 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.962+0000 7f24f3c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:51.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.974+0000 7f24f3c2d640 1 Processor -- start 2024-03-27T08:39:51.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.974+0000 7f24f3c2d640 1 -- start start 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.975+0000 7f24f3c2d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24ec09fd90 0x7f24ec134d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.975+0000 7f24f3c2d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ec0a01a0 con 0x7f24ec09f980 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.975+0000 7f24f3c2d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f24ec06ea70 con 0x7f24ec09fd90 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.975+0000 7f24f11c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24ec09fd90 0x7f24ec134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.975+0000 7f24f19c9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f24ec09f980 0x7f24ec134680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48124/0 (socket says 172.21.15.97:48124) 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.975+0000 7f24f19c9640 1 -- 172.21.15.97:0/4103688790 learned_addr learned my addr 172.21.15.97:0/4103688790 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:39:51.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.976+0000 7f24f11c8640 1 -- 172.21.15.97:0/4103688790 >> v1:172.21.15.97:6789/0 conn(0x7f24ec09f980 legacy=0x7f24ec134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:51.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.976+0000 7f24f11c8640 1 -- 172.21.15.97:0/4103688790 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec09f2e0 con 0x7f24ec09fd90 2024-03-27T08:39:51.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.976+0000 7f24f11c8640 1 --2- 172.21.15.97:0/4103688790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09fd90 0x7f24ec134d90 crc :-1 s=READY pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93dbe78845479b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:51.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.976+0000 7f24f09c7640 1 -- 172.21.15.97:0/4103688790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f24dc009d90 con 0x7f24ec09fd90 2024-03-27T08:39:51.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.976+0000 7f24f09c7640 1 -- 172.21.15.97:0/4103688790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f24dc00e230 con 0x7f24ec09fd90 2024-03-27T08:39:51.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.976+0000 7f24f09c7640 1 -- 172.21.15.97:0/4103688790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f24dc00e4d0 con 0x7f24ec09fd90 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.977+0000 7f24f3c2d640 1 -- 172.21.15.97:0/4103688790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09fd90 msgr2=0x7f24ec134d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.977+0000 7f24f3c2d640 1 --2- 172.21.15.97:0/4103688790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09fd90 0x7f24ec134d90 crc :-1 s=READY pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.977+0000 7f24f3c2d640 1 -- 172.21.15.97:0/4103688790 shutdown_connections 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.977+0000 7f24f3c2d640 1 --2- 172.21.15.97:0/4103688790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09fd90 0x7f24ec134d90 unknown :-1 s=CLOSED pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.977+0000 7f24f3c2d640 1 -- 172.21.15.97:0/4103688790 >> 172.21.15.97:0/4103688790 conn(0x7f24ec0a4bc0 msgr2=0x7f24ec0a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.978+0000 7f24f3c2d640 1 -- 172.21.15.97:0/4103688790 shutdown_connections 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.978+0000 7f24f3c2d640 1 -- 172.21.15.97:0/4103688790 wait complete. 2024-03-27T08:39:51.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.978+0000 7f24f3c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.979+0000 7f24f3c2d640 1 Processor -- start 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.979+0000 7f24f3c2d640 1 -- start start 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.980+0000 7f24f3c2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 0x7f24ec11e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.980+0000 7f24f3c2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1220c0 con 0x7f24ec09f980 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.980+0000 7f24f19c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 0x7f24ec11e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.980+0000 7f24f19c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 0x7f24ec11e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33448/0 (socket says 172.21.15.97:33448) 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.980+0000 7f24f19c9640 1 -- 172.21.15.97:0/95428815 learned_addr learned my addr 172.21.15.97:0/95428815 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24f19c9640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec132250 con 0x7f24ec09f980 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24f19c9640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 0x7f24ec11e000 crc :-1 s=READY pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f24e8005a60 con 0x7f24ec09f980 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f24e8005bc0 con 0x7f24ec09f980 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f24e8005e60 con 0x7f24ec09f980 2024-03-27T08:39:51.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.981+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec131f80 con 0x7f24ec09f980 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.982+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f24e80070b0 con 0x7f24ec09f980 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.982+0000 7f24e27fc640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24d0038750 0x7f24d003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.982+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(698..698 src has 399..698) ==== 3791+0+0 (crc 0 0 0) 0x7f24e80481c0 con 0x7f24ec09f980 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.982+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.982+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=699}) -- 0x7f24d003e960 con 0x7f24ec09f980 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.983+0000 7f24f11c8640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24d0038750 0x7f24d003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:51.976 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.984+0000 7f24f11c8640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24d0038750 0x7f24d003ac60 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:51.984 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:51.992+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:52.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.175+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"} v 0) -- 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:52.828 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.836+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v699) ==== 221+0+0 (crc 0 0 0) 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:52.828 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.836+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(699..699 src has 399..699) ==== 628+0+0 (crc 0 0 0) 0x7fc1c0009df0 con 0x13450a0 2024-03-27T08:39:52.828 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.836+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(699..699 src has 399..699) ==== 628+0+0 (crc 0 0 0) 0x7f24e800d330 con 0x7f24ec09f980 2024-03-27T08:39:52.828 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.836+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=700}) -- 0x7fc1b803daa0 con 0x13450a0 2024-03-27T08:39:52.828 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.836+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=700}) -- 0x7f24d003f8c0 con 0x7f24ec09f980 2024-03-27T08:39:52.828 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.836+0000 7fc1d1ffb640 1 -- 172.21.15.97:0/3656451783 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 200.6 200:6b5d76a5:::two:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e699) -- 0x7fc1b803dcf0 con 0x134c230 2024-03-27T08:39:52.942 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:52.949+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"} v 0) -- 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.019+0000 7fc1d919b640 1 -- 172.21.15.97:0/3656451783 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 two [writefull 0~11812] v699'2 uv2 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7fc1cc00a430 con 0x134c230 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.019+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 >> v2:172.21.15.97:6805/1668842645 conn(0x134c230 msgr2=0x1476920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.019+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> v2:172.21.15.97:6805/1668842645 conn(0x134c230 0x1476920 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.019+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc1b8036fe0 msgr2=0x7fc1b80394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.019+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc1b8036fe0 0x7fc1b80394f0 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 msgr2=0x13ddb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13ddb10 crc :-1 s=READY pgs=12924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 shutdown_connections 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> v2:172.21.15.97:6805/1668842645 conn(0x134c230 0x1476920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.012 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc1b8036fe0 0x7fc1b80394f0 unknown :-1 s=CLOSED pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.013 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 --2- 172.21.15.97:0/3656451783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x13450a0 0x13ddb10 unknown :-1 s=CLOSED pgs=12924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.013 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 >> 172.21.15.97:0/3656451783 conn(0x1265ea0 msgr2=0x134ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:53.013 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 shutdown_connections 2024-03-27T08:39:53.013 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.020+0000 7fc1e1034ec0 1 -- 172.21.15.97:0/3656451783 wait complete. 2024-03-27T08:39:53.859 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.867+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v700) ==== 221+0+0 (crc 0 0 0) 0x7f24ec069180 con 0x7f24ec09f980 2024-03-27T08:39:53.859 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:39:53.861 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.869+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24d0038750 msgr2=0x7f24d003ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.869+0000 7f24f3c2d640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24d0038750 0x7f24d003ac60 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.869+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 msgr2=0x7f24ec11e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.869+0000 7f24f3c2d640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 0x7f24ec11e000 crc :-1 s=READY pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.869+0000 7f24e27fc640 1 -- 172.21.15.97:0/95428815 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(700..700 src has 399..700) ==== 628+0+0 (crc 0 0 0) 0x7f24e800ec20 con 0x7f24ec09f980 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.870+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 shutdown_connections 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.870+0000 7f24f3c2d640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24d0038750 0x7f24d003ac60 unknown :-1 s=CLOSED pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.870+0000 7f24f3c2d640 1 --2- 172.21.15.97:0/95428815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ec09f980 0x7f24ec11e000 unknown :-1 s=CLOSED pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.870+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 >> 172.21.15.97:0/95428815 conn(0x7f24ec0a4bc0 msgr2=0x7f24ec0a16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.870+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 shutdown_connections 2024-03-27T08:39:53.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.870+0000 7f24f3c2d640 1 -- 172.21.15.97:0/95428815 wait complete. 2024-03-27T08:39:53.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 99489 2024-03-27T08:39:53.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -ne 0 ']' 2024-03-27T08:39:53.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ true 2024-03-27T08:39:53.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put three /etc/passwd 2024-03-27T08:39:53.911 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.919+0000 7f694c52dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:53.916 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.925+0000 7f694c52dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:53.917 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.925+0000 7f694c52dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:53.918 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.926+0000 7f694c52dec0 1 Processor -- start 2024-03-27T08:39:53.918 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.926+0000 7f694c52dec0 1 -- start start 2024-03-27T08:39:53.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.928+0000 7f694c52dec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x18021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:53.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.928+0000 7f694c52dec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x16d80b0 con 0x1772f70 2024-03-27T08:39:53.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.928+0000 7f693ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x18021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:53.920 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.928+0000 7f693ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x18021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33464/0 (socket says 172.21.15.97:33464) 2024-03-27T08:39:53.921 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.928+0000 7f693ffff640 1 -- 172.21.15.97:0/2172385216 learned_addr learned my addr 172.21.15.97:0/2172385216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:53.921 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.928+0000 7f693ffff640 1 -- 172.21.15.97:0/2172385216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x16781e0 con 0x1772f70 2024-03-27T08:39:53.921 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.929+0000 7f693ffff640 1 --2- 172.21.15.97:0/2172385216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x18021d0 crc :-1 s=READY pgs=12933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d981b8ee4128458 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:53.921 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.929+0000 7f693e7fc640 1 -- 172.21.15.97:0/2172385216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6930033040 con 0x1772f70 2024-03-27T08:39:53.921 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.929+0000 7f693e7fc640 1 -- 172.21.15.97:0/2172385216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6930034040 con 0x1772f70 2024-03-27T08:39:53.921 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.929+0000 7f693e7fc640 1 -- 172.21.15.97:0/2172385216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f693002ecb0 con 0x1772f70 2024-03-27T08:39:53.922 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 -- 172.21.15.97:0/2172385216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 msgr2=0x18021d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:53.922 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 --2- 172.21.15.97:0/2172385216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x18021d0 crc :-1 s=READY pgs=12933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.922 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 -- 172.21.15.97:0/2172385216 shutdown_connections 2024-03-27T08:39:53.922 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 --2- 172.21.15.97:0/2172385216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x18021d0 unknown :-1 s=CLOSED pgs=12933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:53.922 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 -- 172.21.15.97:0/2172385216 >> 172.21.15.97:0/2172385216 conn(0x168cea0 msgr2=0x176bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:53.922 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 -- 172.21.15.97:0/2172385216 shutdown_connections 2024-03-27T08:39:53.923 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.930+0000 7f694c52dec0 1 -- 172.21.15.97:0/2172385216 wait complete. 2024-03-27T08:39:53.923 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.931+0000 7f694c52dec0 1 Processor -- start 2024-03-27T08:39:53.923 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.931+0000 7f694c52dec0 1 -- start start 2024-03-27T08:39:53.924 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f694c52dec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x176bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:53.924 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f694c52dec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x16d5b30 con 0x1772f70 2024-03-27T08:39:53.924 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f693ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x176bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:53.924 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f693ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x176bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33468/0 (socket says 172.21.15.97:33468) 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f693ffff640 1 -- 172.21.15.97:0/3293756561 learned_addr learned my addr 172.21.15.97:0/3293756561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f693ffff640 1 -- 172.21.15.97:0/3293756561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x166ae30 con 0x1772f70 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.932+0000 7f693ffff640 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x176bd50 crc :-1 s=READY pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.933+0000 7f693d7fa640 1 -- 172.21.15.97:0/3293756561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6930009620 con 0x1772f70 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.933+0000 7f693d7fa640 1 -- 172.21.15.97:0/3293756561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6930009780 con 0x1772f70 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.933+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x166a890 con 0x1772f70 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.933+0000 7f693d7fa640 1 -- 172.21.15.97:0/3293756561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6930009a20 con 0x1772f70 2024-03-27T08:39:53.925 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.933+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x176b900 con 0x1772f70 2024-03-27T08:39:53.926 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.934+0000 7f693d7fa640 1 -- 172.21.15.97:0/3293756561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f6930009c20 con 0x1772f70 2024-03-27T08:39:53.926 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.934+0000 7f693d7fa640 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f692c036fe0 0x7f692c0394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:53.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.934+0000 7f693d7fa640 1 -- 172.21.15.97:0/3293756561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(700..700 src has 399..700) ==== 3791+0+0 (crc 0 0 0) 0x7f693006bb40 con 0x1772f70 2024-03-27T08:39:53.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.934+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> v2:172.21.15.190:6805/971408905 conn(0x1804360 0x1898ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:53.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.934+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 200.3 200:c57798d7:::three:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e700) -- 0x188ab50 con 0x1804360 2024-03-27T08:39:53.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.935+0000 7f693f7fe640 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f692c036fe0 0x7f692c0394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:53.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.935+0000 7f693f7fe640 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f692c036fe0 0x7f692c0394f0 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:53.927 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.935+0000 7f694499b640 1 --2- 172.21.15.97:0/3293756561 >> v2:172.21.15.190:6805/971408905 conn(0x1804360 0x1898ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:39:53.930 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:53.939+0000 7f694499b640 1 --2- 172.21.15.97:0/3293756561 >> v2:172.21.15.190:6805/971408905 conn(0x1804360 0x1898ec0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.123+0000 7f694499b640 1 -- 172.21.15.97:0/3293756561 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 three [writefull 0~11812] v700'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f693800a430 con 0x1804360 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 >> v2:172.21.15.190:6805/971408905 conn(0x1804360 msgr2=0x1898ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> v2:172.21.15.190:6805/971408905 conn(0x1804360 0x1898ec0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f692c036fe0 msgr2=0x7f692c0394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f692c036fe0 0x7f692c0394f0 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 msgr2=0x176bd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:54.116 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x176bd50 crc :-1 s=READY pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 shutdown_connections 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> v2:172.21.15.190:6805/971408905 conn(0x1804360 0x1898ec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f692c036fe0 0x7f692c0394f0 unknown :-1 s=CLOSED pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 --2- 172.21.15.97:0/3293756561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1772f70 0x176bd50 unknown :-1 s=CLOSED pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 >> 172.21.15.97:0/3293756561 conn(0x168cea0 msgr2=0x1801de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 shutdown_connections 2024-03-27T08:39:54.117 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.124+0000 7f694c52dec0 1 -- 172.21.15.97:0/3293756561 wait complete. 2024-03-27T08:39:54.118 INFO:tasks.workunit.client.0.smithi097.stderr:++ uuidgen 2024-03-27T08:39:54.119 INFO:tasks.workunit.client.0.smithi097.stderr:+ pp=7165a426-c096-42b4-86ea-e81ff2d1e62a 2024-03-27T08:39:54.119 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool create 7165a426-c096-42b4-86ea-e81ff2d1e62a 12 2024-03-27T08:39:54.210 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.211+0000 7f76558ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:54.211 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.219+0000 7f76558ed640 1 Processor -- start 2024-03-27T08:39:54.211 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.219+0000 7f76558ed640 1 -- start start 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.220+0000 7f76558ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f765009fee0 0x7f765006d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.220+0000 7f76558ed640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765009fd00 con 0x7f765006d780 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.220+0000 7f76558ed640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f765006e720 con 0x7f765009fee0 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f765009fee0 0x7f765006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f765009fee0 0x7f765006d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33480/0 (socket says 172.21.15.97:33480) 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764e7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f765006d780 0x7f7650130fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48128/0 (socket says 172.21.15.97:48128) 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764effd640 1 -- 172.21.15.97:0/2980592800 learned_addr learned my addr 172.21.15.97:0/2980592800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764effd640 1 -- 172.21.15.97:0/2980592800 >> v1:172.21.15.97:6789/0 conn(0x7f765006d780 legacy=0x7f7650130fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764effd640 1 -- 172.21.15.97:0/2980592800 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765009f9f0 con 0x7f765009fee0 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764effd640 1 --2- 172.21.15.97:0/2980592800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765009fee0 0x7f765006d240 crc :-1 s=READY pgs=12936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98dd7fe988b6cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:54.213 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764dffb640 1 -- 172.21.15.97:0/2980592800 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f764000ade0 con 0x7f765009fee0 2024-03-27T08:39:54.214 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764dffb640 1 -- 172.21.15.97:0/2980592800 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f764000f340 con 0x7f765009fee0 2024-03-27T08:39:54.214 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.221+0000 7f764dffb640 1 -- 172.21.15.97:0/2980592800 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f764000f5e0 con 0x7f765009fee0 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 -- 172.21.15.97:0/2980592800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765009fee0 msgr2=0x7f765006d240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 --2- 172.21.15.97:0/2980592800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765009fee0 0x7f765006d240 crc :-1 s=READY pgs=12936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 -- 172.21.15.97:0/2980592800 shutdown_connections 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 --2- 172.21.15.97:0/2980592800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765009fee0 0x7f765006d240 unknown :-1 s=CLOSED pgs=12936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 -- 172.21.15.97:0/2980592800 >> 172.21.15.97:0/2980592800 conn(0x7f765009aa00 msgr2=0x7f765009d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 -- 172.21.15.97:0/2980592800 shutdown_connections 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 1 -- 172.21.15.97:0/2980592800 wait complete. 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.222+0000 7f76558ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:54.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.223+0000 7f76558ed640 1 Processor -- start 2024-03-27T08:39:54.216 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.224+0000 7f76558ed640 1 -- start start 2024-03-27T08:39:54.216 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f76558ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 0x7f76501a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:54.216 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f76558ed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76501a6030 con 0x7f765006d780 2024-03-27T08:39:54.216 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f764effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 0x7f76501a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:54.216 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f764effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 0x7f76501a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33492/0 (socket says 172.21.15.97:33492) 2024-03-27T08:39:54.217 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f764effd640 1 -- 172.21.15.97:0/2246617077 learned_addr learned my addr 172.21.15.97:0/2246617077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:54.217 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f764effd640 1 -- 172.21.15.97:0/2246617077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76501b6200 con 0x7f765006d780 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f764effd640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 0x7f76501a21d0 crc :-1 s=READY pgs=12937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7640012430 con 0x7f765006d780 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7640012590 con 0x7f765006d780 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76501b6430 con 0x7f765006d780 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7640012830 con 0x7f765006d780 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.225+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76501a4d90 con 0x7f765006d780 2024-03-27T08:39:54.218 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.226+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f76400129e0 con 0x7f765006d780 2024-03-27T08:39:54.219 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.226+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76501b6620 con 0x7f765006d780 2024-03-27T08:39:54.219 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.226+0000 7f7637fff640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f763c0387a0 0x7f763c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:54.219 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.226+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(700..700 src has 399..700) ==== 3791+0+0 (crc 0 0 0) 0x7f7640049510 con 0x7f765006d780 2024-03-27T08:39:54.219 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.227+0000 7f764e7fc640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f763c0387a0 0x7f763c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:54.219 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.227+0000 7f764e7fc640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f763c0387a0 0x7f763c03acb0 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:54.223 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.232+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f764000cb70 con 0x7f765006d780 2024-03-27T08:39:54.397 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.406+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pg_num": 12} v 0) -- 0x7f76500693c0 con 0x7f765006d780 2024-03-27T08:39:54.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:54.877+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pg_num": 12}]=0 pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' created v701) ==== 176+0+0 (crc 0 0 0) 0x7f764000c760 con 0x7f765006d780 2024-03-27T08:39:55.013 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.022+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pg_num": 12} v 0) -- 0x7f76501a4a30 con 0x7f765006d780 2024-03-27T08:39:55.014 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.023+0000 7f7637fff640 1 -- 172.21.15.97:0/2246617077 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pg_num": 12}]=0 pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' already exists v701) ==== 183+0+0 (crc 0 0 0) 0x7f764000c940 con 0x7f765006d780 2024-03-27T08:39:55.014 INFO:tasks.workunit.client.0.smithi097.stderr:pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' already exists 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f763c0387a0 msgr2=0x7f763c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f763c0387a0 0x7f763c03acb0 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 msgr2=0x7f76501a21d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 0x7f76501a21d0 crc :-1 s=READY pgs=12937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 shutdown_connections 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f763c0387a0 0x7f763c03acb0 unknown :-1 s=CLOSED pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:55.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 --2- 172.21.15.97:0/2246617077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f765006d780 0x7f76501a21d0 unknown :-1 s=CLOSED pgs=12937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:55.018 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 >> 172.21.15.97:0/2246617077 conn(0x7f765009aa00 msgr2=0x7f765009bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:55.018 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 shutdown_connections 2024-03-27T08:39:55.018 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.025+0000 7f76558ed640 1 -- 172.21.15.97:0/2246617077 wait complete. 2024-03-27T08:39:55.033 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool application enable 7165a426-c096-42b4-86ea-e81ff2d1e62a rados 2024-03-27T08:39:55.158 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.156+0000 7fb40da20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:55.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.168+0000 7fb40da20640 1 Processor -- start 2024-03-27T08:39:55.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.168+0000 7fb40da20640 1 -- start start 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.169+0000 7fb40da20640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb408137b10 0x7fb408139f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.169+0000 7fb40da20640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40806eb50 con 0x7fb40809f6e0 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb40da20640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb40809f2f0 con 0x7fb408137b10 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb406ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb40809f6e0 0x7fb40809faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48142/0 (socket says 172.21.15.97:48142) 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb406ffd640 1 -- 172.21.15.97:0/287204146 learned_addr learned my addr 172.21.15.97:0/287204146 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb4067fc640 1 --2- 172.21.15.97:0/287204146 >> v2:172.21.15.97:3300/0 conn(0x7fb408137b10 0x7fb408139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb405ffb640 1 -- 172.21.15.97:0/287204146 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278459560 0 0) 0x7fb40806eb50 con 0x7fb40809f6e0 2024-03-27T08:39:55.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb405ffb640 1 -- 172.21.15.97:0/287204146 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3f0003580 con 0x7fb40809f6e0 2024-03-27T08:39:55.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb4067fc640 1 -- 172.21.15.97:0/287204146 >> v1:172.21.15.97:6789/0 conn(0x7fb40809f6e0 legacy=0x7fb40809faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:55.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb4067fc640 1 -- 172.21.15.97:0/287204146 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4080a0200 con 0x7fb408137b10 2024-03-27T08:39:55.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb4067fc640 1 --2- 172.21.15.97:0/287204146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408137b10 0x7fb408139f30 crc :-1 s=READY pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378f675bf158b0e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:55.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.170+0000 7fb405ffb640 1 -- 172.21.15.97:0/287204146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3fc009d90 con 0x7fb408137b10 2024-03-27T08:39:55.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.171+0000 7fb405ffb640 1 -- 172.21.15.97:0/287204146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb3fc00e230 con 0x7fb408137b10 2024-03-27T08:39:55.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.171+0000 7fb405ffb640 1 -- 172.21.15.97:0/287204146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3fc00e4d0 con 0x7fb408137b10 2024-03-27T08:39:55.164 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 -- 172.21.15.97:0/287204146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408137b10 msgr2=0x7fb408139f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:55.164 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 --2- 172.21.15.97:0/287204146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408137b10 0x7fb408139f30 crc :-1 s=READY pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:55.164 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 -- 172.21.15.97:0/287204146 shutdown_connections 2024-03-27T08:39:55.164 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 --2- 172.21.15.97:0/287204146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408137b10 0x7fb408139f30 unknown :-1 s=CLOSED pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:55.164 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 -- 172.21.15.97:0/287204146 >> 172.21.15.97:0/287204146 conn(0x7fb4080a1860 msgr2=0x7fb4080a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:55.165 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 -- 172.21.15.97:0/287204146 shutdown_connections 2024-03-27T08:39:55.165 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.172+0000 7fb40da20640 1 -- 172.21.15.97:0/287204146 wait complete. 2024-03-27T08:39:55.165 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.173+0000 7fb40da20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:55.167 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.175+0000 7fb40da20640 1 Processor -- start 2024-03-27T08:39:55.167 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.175+0000 7fb40da20640 1 -- start start 2024-03-27T08:39:55.173 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.177+0000 7fb40da20640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 0x7fb40811f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:55.173 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.177+0000 7fb40da20640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb408120780 con 0x7fb40809f6e0 2024-03-27T08:39:55.173 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.177+0000 7fb406ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 0x7fb40811f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:55.173 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.177+0000 7fb406ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 0x7fb40811f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33510/0 (socket says 172.21.15.97:33510) 2024-03-27T08:39:55.174 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.177+0000 7fb406ffd640 1 -- 172.21.15.97:0/1590098758 learned_addr learned my addr 172.21.15.97:0/1590098758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:55.174 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.178+0000 7fb406ffd640 1 -- 172.21.15.97:0/1590098758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40811f150 con 0x7fb40809f6e0 2024-03-27T08:39:55.174 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.178+0000 7fb406ffd640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 0x7fb40811f5b0 crc :-1 s=READY pgs=12940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:55.174 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.178+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3f8005a60 con 0x7fb40809f6e0 2024-03-27T08:39:55.174 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.178+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb408121ea0 con 0x7fb40809f6e0 2024-03-27T08:39:55.175 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.178+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb3f8005bc0 con 0x7fb40809f6e0 2024-03-27T08:39:55.175 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.178+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb3f8005e60 con 0x7fb40809f6e0 2024-03-27T08:39:55.175 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.179+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb408122780 con 0x7fb40809f6e0 2024-03-27T08:39:55.175 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.180+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fb3f800c7a0 con 0x7fb40809f6e0 2024-03-27T08:39:55.175 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.180+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4080693c0 con 0x7fb40809f6e0 2024-03-27T08:39:55.176 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.180+0000 7fb3effff640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fb3f4038750 0x7fb3f403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:55.176 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.180+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(701..701 src has 399..701) ==== 4166+0+0 (crc 0 0 0) 0x7fb3f80486d0 con 0x7fb40809f6e0 2024-03-27T08:39:55.176 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.180+0000 7fb4067fc640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fb3f4038750 0x7fb3f403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:55.176 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.181+0000 7fb4067fc640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fb3f4038750 0x7fb3f403ac60 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:55.181 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.189+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fb3f800d510 con 0x7fb40809f6e0 2024-03-27T08:39:55.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.370+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "app": "rados"} v 0) -- 0x7fb408075190 con 0x7fb40809f6e0 2024-03-27T08:39:55.872 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:55.881+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "app": "rados"}]=0 enabled application 'rados' on pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' v702) ==== 213+0+0 (crc 0 0 0) 0x7fb3f800d150 con 0x7fb40809f6e0 2024-03-27T08:39:56.086 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.095+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "app": "rados"} v 0) -- 0x7fb408122a30 con 0x7fb40809f6e0 2024-03-27T08:39:56.876 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.884+0000 7fb3effff640 1 -- 172.21.15.97:0/1590098758 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "app": "rados"}]=0 enabled application 'rados' on pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' v703) ==== 213+0+0 (crc 0 0 0) 0x7fb408122a30 con 0x7fb40809f6e0 2024-03-27T08:39:56.876 INFO:tasks.workunit.client.0.smithi097.stderr:enabled application 'rados' on pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fb3f4038750 msgr2=0x7fb3f403ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fb3f4038750 0x7fb3f403ac60 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 msgr2=0x7fb40811f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 0x7fb40811f5b0 crc :-1 s=READY pgs=12940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 shutdown_connections 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fb3f4038750 0x7fb3f403ac60 unknown :-1 s=CLOSED pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:56.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 --2- 172.21.15.97:0/1590098758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb40809f6e0 0x7fb40811f5b0 unknown :-1 s=CLOSED pgs=12940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:56.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 >> 172.21.15.97:0/1590098758 conn(0x7fb4080a1860 msgr2=0x7fb408134610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:56.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 shutdown_connections 2024-03-27T08:39:56.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.887+0000 7fb40da20640 1 -- 172.21.15.97:0/1590098758 wait complete. 2024-03-27T08:39:56.890 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 7165a426-c096-42b4-86ea-e81ff2d1e62a max_objects 10 2024-03-27T08:39:56.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.989+0000 7f960f26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:56.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.998+0000 7f960f26d640 1 Processor -- start 2024-03-27T08:39:56.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.998+0000 7f960f26d640 1 -- start start 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960f26d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9608135820 0x7f9608137c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960f26d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9608073400 con 0x7f960809b540 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960f26d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f960809c510 con 0x7f9608135820 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960c808640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9608135820 0x7f9608137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960d009640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f960809b540 0x7f960809b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48144/0 (socket says 172.21.15.97:48144) 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960c808640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9608135820 0x7f9608137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33526/0 (socket says 172.21.15.97:33526) 2024-03-27T08:39:56.993 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960d009640 1 -- 172.21.15.97:0/675677611 learned_addr learned my addr 172.21.15.97:0/675677611 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960c808640 1 -- 172.21.15.97:0/675677611 >> v1:172.21.15.97:6789/0 conn(0x7f960809b540 legacy=0x7f960809b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:56.999+0000 7f960c808640 1 -- 172.21.15.97:0/675677611 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96080735c0 con 0x7f9608135820 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f960c808640 1 --2- 172.21.15.97:0/675677611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9608135820 0x7f9608137c40 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a50a75760f38908 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f95fffff640 1 -- 172.21.15.97:0/675677611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95f8009d80 con 0x7f9608135820 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f95fffff640 1 -- 172.21.15.97:0/675677611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f95f800e230 con 0x7f9608135820 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f95fffff640 1 -- 172.21.15.97:0/675677611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95f800e4d0 con 0x7f9608135820 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f960f26d640 1 -- 172.21.15.97:0/675677611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9608135820 msgr2=0x7f9608137c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:56.994 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f960f26d640 1 --2- 172.21.15.97:0/675677611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9608135820 0x7f9608137c40 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:56.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f960f26d640 1 -- 172.21.15.97:0/675677611 shutdown_connections 2024-03-27T08:39:56.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f960f26d640 1 --2- 172.21.15.97:0/675677611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9608135820 0x7f9608137c40 unknown :-1 s=CLOSED pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:56.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.000+0000 7f960f26d640 1 -- 172.21.15.97:0/675677611 >> 172.21.15.97:0/675677611 conn(0x7f96080a0b50 msgr2=0x7f96080a0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:56.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.001+0000 7f960f26d640 1 -- 172.21.15.97:0/675677611 shutdown_connections 2024-03-27T08:39:56.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.001+0000 7f960f26d640 1 -- 172.21.15.97:0/675677611 wait complete. 2024-03-27T08:39:56.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.001+0000 7f960f26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:39:56.999 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.002+0000 7f960f26d640 1 Processor -- start 2024-03-27T08:39:56.999 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.002+0000 7f960f26d640 1 -- start start 2024-03-27T08:39:56.999 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.003+0000 7f960f26d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 0x7f96081c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:56.999 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.003+0000 7f960f26d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f960811e500 con 0x7f960809b540 2024-03-27T08:39:56.999 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.003+0000 7f960d009640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 0x7f96081c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:56.999 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.003+0000 7f960d009640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 0x7f96081c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33532/0 (socket says 172.21.15.97:33532) 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.003+0000 7f960d009640 1 -- 172.21.15.97:0/2722472901 learned_addr learned my addr 172.21.15.97:0/2722472901 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.003+0000 7f960d009640 1 -- 172.21.15.97:0/2722472901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96081c3cc0 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.004+0000 7f960d009640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 0x7f96081c4060 crc :-1 s=READY pgs=12943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.004+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9604005a60 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.004+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9604005bc0 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.004+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9604005e60 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.004+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9608120980 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.004+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960811f600 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.005+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f96040070b0 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.005+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96080693c0 con 0x7f960809b540 2024-03-27T08:39:57.000 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.005+0000 7f95fdffb640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95ec0387a0 0x7f95ec03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:39:57.001 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.005+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(703..703 src has 399..703) ==== 4179+0+0 (crc 0 0 0) 0x7f96040482f0 con 0x7f960809b540 2024-03-27T08:39:57.001 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.005+0000 7f960c808640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95ec0387a0 0x7f95ec03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:39:57.001 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.006+0000 7f960c808640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95ec0387a0 0x7f95ec03acb0 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:39:57.002 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.010+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f960400d540 con 0x7f960809b540 2024-03-27T08:39:57.238 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.247+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "field": "max_objects", "val": "10"} v 0) -- 0x7f9608073400 con 0x7f960809b540 2024-03-27T08:39:57.829 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.837+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 7165a426-c096-42b4-86ea-e81ff2d1e62a v704) ==== 223+0+0 (crc 0 0 0) 0x7f9608073400 con 0x7f960809b540 2024-03-27T08:39:57.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:57.982+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "field": "max_objects", "val": "10"} v 0) -- 0x7f9608073400 con 0x7f960809b540 2024-03-27T08:39:58.881 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.890+0000 7f95fdffb640 1 -- 172.21.15.97:0/2722472901 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 7165a426-c096-42b4-86ea-e81ff2d1e62a v705) ==== 223+0+0 (crc 0 0 0) 0x7f9608073400 con 0x7f960809b540 2024-03-27T08:39:58.881 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 10 for pool 7165a426-c096-42b4-86ea-e81ff2d1e62a 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95ec0387a0 msgr2=0x7f95ec03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95ec0387a0 0x7f95ec03acb0 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 msgr2=0x7f96081c4060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 0x7f96081c4060 crc :-1 s=READY pgs=12943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 shutdown_connections 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f95ec0387a0 0x7f95ec03acb0 unknown :-1 s=CLOSED pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 --2- 172.21.15.97:0/2722472901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960809b540 0x7f96081c4060 unknown :-1 s=CLOSED pgs=12943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 >> 172.21.15.97:0/2722472901 conn(0x7f96080a0b50 msgr2=0x7f96081302f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 shutdown_connections 2024-03-27T08:39:58.885 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:39:58.893+0000 7f960f26d640 1 -- 172.21.15.97:0/2722472901 wait complete. 2024-03-27T08:39:58.896 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-27T08:40:28.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ seq 1 10 2024-03-27T08:40:28.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:28.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj1 /etc/passwd 2024-03-27T08:40:28.940 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.948+0000 7fc646946ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:28.945 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.954+0000 7fc646946ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:28.945 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.954+0000 7fc646946ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:28.946 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.955+0000 7fc646946ec0 1 Processor -- start 2024-03-27T08:40:28.946 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.955+0000 7fc646946ec0 1 -- start start 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc646946ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x185c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc646946ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x16a80b0 con 0x1742ad0 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63ed9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x185c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63ed9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x185c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46100/0 (socket says 172.21.15.97:46100) 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63ed9b640 1 -- 172.21.15.97:0/3584547105 learned_addr learned my addr 172.21.15.97:0/3584547105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63ed9b640 1 -- 172.21.15.97:0/3584547105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x16481e0 con 0x1742ad0 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63ed9b640 1 --2- 172.21.15.97:0/3584547105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x185c8a0 crc :-1 s=READY pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9aa71b3c392a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63d598640 1 -- 172.21.15.97:0/3584547105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc62802fb20 con 0x1742ad0 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63d598640 1 -- 172.21.15.97:0/3584547105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc62802fc80 con 0x1742ad0 2024-03-27T08:40:28.948 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.956+0000 7fc63d598640 1 -- 172.21.15.97:0/3584547105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc628035410 con 0x1742ad0 2024-03-27T08:40:28.951 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 -- 172.21.15.97:0/3584547105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 msgr2=0x185c8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:28.951 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 --2- 172.21.15.97:0/3584547105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x185c8a0 crc :-1 s=READY pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 -- 172.21.15.97:0/3584547105 shutdown_connections 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 --2- 172.21.15.97:0/3584547105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x185c8a0 unknown :-1 s=CLOSED pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 -- 172.21.15.97:0/3584547105 >> 172.21.15.97:0/3584547105 conn(0x165cea0 msgr2=0x1746df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 -- 172.21.15.97:0/3584547105 shutdown_connections 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.957+0000 7fc646946ec0 1 -- 172.21.15.97:0/3584547105 wait complete. 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.958+0000 7fc646946ec0 1 Processor -- start 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.958+0000 7fc646946ec0 1 -- start start 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.959+0000 7fc646946ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x17d5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.959+0000 7fc646946ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x16a5b30 con 0x1742ad0 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.959+0000 7fc63ed9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x17d5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.959+0000 7fc63ed9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x17d5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46102/0 (socket says 172.21.15.97:46102) 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.959+0000 7fc63ed9b640 1 -- 172.21.15.97:0/2377508022 learned_addr learned my addr 172.21.15.97:0/2377508022 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc63ed9b640 1 -- 172.21.15.97:0/2377508022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x163ae30 con 0x1742ad0 2024-03-27T08:40:28.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc63ed9b640 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x17d5220 crc :-1 s=READY pgs=12947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:28.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc6377fe640 1 -- 172.21.15.97:0/2377508022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc62803a040 con 0x1742ad0 2024-03-27T08:40:28.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc6377fe640 1 -- 172.21.15.97:0/2377508022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc62802f9f0 con 0x1742ad0 2024-03-27T08:40:28.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x163a890 con 0x1742ad0 2024-03-27T08:40:28.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc6377fe640 1 -- 172.21.15.97:0/2377508022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc628036cb0 con 0x1742ad0 2024-03-27T08:40:28.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.960+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x17c3f30 con 0x1742ad0 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.961+0000 7fc6377fe640 1 -- 172.21.15.97:0/2377508022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fc62803d040 con 0x1742ad0 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.961+0000 7fc6377fe640 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc624037030 0x7fc624039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.961+0000 7fc6377fe640 1 -- 172.21.15.97:0/2377508022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7fc62806c270 con 0x1742ad0 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.961+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> v2:172.21.15.190:6805/971408905 conn(0x17d5880 0x17c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.961+0000 7fc63e59a640 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc624037030 0x7fc624039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.961+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 201.7 201:ff7b1f36:::obj1:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x17c45f0 con 0x17d5880 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.962+0000 7fc63e59a640 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc624037030 0x7fc624039540 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:28.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.962+0000 7fc63f59c640 1 --2- 172.21.15.97:0/2377508022 >> v2:172.21.15.190:6805/971408905 conn(0x17d5880 0x17c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:28.957 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.966+0000 7fc63f59c640 1 --2- 172.21.15.97:0/2377508022 >> v2:172.21.15.190:6805/971408905 conn(0x17d5880 0x17c1530 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:28.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc63f59c640 1 -- 172.21.15.97:0/2377508022 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj1 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fc63000c430 con 0x17d5880 2024-03-27T08:40:28.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 >> v2:172.21.15.190:6805/971408905 conn(0x17d5880 msgr2=0x17c1530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:28.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> v2:172.21.15.190:6805/971408905 conn(0x17d5880 0x17c1530 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc624037030 msgr2=0x7fc624039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:28.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc624037030 0x7fc624039540 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 msgr2=0x17d5220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x17d5220 crc :-1 s=READY pgs=12947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 shutdown_connections 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> v2:172.21.15.190:6805/971408905 conn(0x17d5880 0x17c1530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fc624037030 0x7fc624039540 unknown :-1 s=CLOSED pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 --2- 172.21.15.97:0/2377508022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1742ad0 0x17d5220 unknown :-1 s=CLOSED pgs=12947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 >> 172.21.15.97:0/2377508022 conn(0x165cea0 msgr2=0x1746df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 shutdown_connections 2024-03-27T08:40:28.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:28.979+0000 7fc646946ec0 1 -- 172.21.15.97:0/2377508022 wait complete. 2024-03-27T08:40:28.973 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:28.973 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj2 /etc/passwd 2024-03-27T08:40:29.011 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.020+0000 7fa125320ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.016 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.025+0000 7fa125320ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.016 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.025+0000 7fa125320ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.026+0000 7fa125320ec0 1 Processor -- start 2024-03-27T08:40:29.017 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.026+0000 7fa125320ec0 1 -- start start 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa125320ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa125320ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xd740b0 con 0xdf48f0 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa11d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa11d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46114/0 (socket says 172.21.15.97:46114) 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa11d79b640 1 -- 172.21.15.97:0/3807870694 learned_addr learned my addr 172.21.15.97:0/3807870694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa11d79b640 1 -- 172.21.15.97:0/3807870694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xd141e0 con 0xdf48f0 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa11d79b640 1 --2- 172.21.15.97:0/3807870694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe091e0 crc :-1 s=READY pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b399b62192da29 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa116ffd640 1 -- 172.21.15.97:0/3807870694 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa108029a90 con 0xdf48f0 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa116ffd640 1 -- 172.21.15.97:0/3807870694 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa108029bf0 con 0xdf48f0 2024-03-27T08:40:29.019 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.027+0000 7fa116ffd640 1 -- 172.21.15.97:0/3807870694 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa108006790 con 0xdf48f0 2024-03-27T08:40:29.020 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 -- 172.21.15.97:0/3807870694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 msgr2=0xe091e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 --2- 172.21.15.97:0/3807870694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe091e0 crc :-1 s=READY pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 -- 172.21.15.97:0/3807870694 shutdown_connections 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 --2- 172.21.15.97:0/3807870694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe091e0 unknown :-1 s=CLOSED pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 -- 172.21.15.97:0/3807870694 >> 172.21.15.97:0/3807870694 conn(0xd28ea0 msgr2=0xe9e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 -- 172.21.15.97:0/3807870694 shutdown_connections 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.028+0000 7fa125320ec0 1 -- 172.21.15.97:0/3807870694 wait complete. 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.029+0000 7fa125320ec0 1 Processor -- start 2024-03-27T08:40:29.021 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.029+0000 7fa125320ec0 1 -- start start 2024-03-27T08:40:29.023 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa125320ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe9d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.023 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa125320ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0xd71b30 con 0xdf48f0 2024-03-27T08:40:29.023 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa11d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe9d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.023 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa11d79b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe9d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46122/0 (socket says 172.21.15.97:46122) 2024-03-27T08:40:29.023 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa11d79b640 1 -- 172.21.15.97:0/655489830 learned_addr learned my addr 172.21.15.97:0/655489830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa11d79b640 1 -- 172.21.15.97:0/655489830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xd06e30 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa11d79b640 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe9d390 crc :-1 s=READY pgs=12951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.030+0000 7fa115ffb640 1 -- 172.21.15.97:0/655489830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa10803a040 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.031+0000 7fa115ffb640 1 -- 172.21.15.97:0/655489830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa108006e70 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.031+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xd06890 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.031+0000 7fa115ffb640 1 -- 172.21.15.97:0/655489830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa108035580 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.031+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0xe8bf80 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.031+0000 7fa115ffb640 1 -- 172.21.15.97:0/655489830 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fa10803d040 con 0xdf48f0 2024-03-27T08:40:29.024 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.032+0000 7fa115ffb640 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa104037030 0x7fa104039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.032+0000 7fa115ffb640 1 -- 172.21.15.97:0/655489830 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7fa10802e040 con 0xdf48f0 2024-03-27T08:40:29.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.032+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> v2:172.21.15.190:6809/239686547 conn(0xdf5240 0xea5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.032+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 --> v2:172.21.15.190:6809/239686547 -- osd_op(unknown.0.0:1 201.8 201:104778fc:::obj2:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0xe8c520 con 0xdf5240 2024-03-27T08:40:29.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.032+0000 7fa11cf9a640 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa104037030 0x7fa104039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.033+0000 7fa11cf9a640 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa104037030 0x7fa104039540 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.025 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.033+0000 7fa11df9c640 1 --2- 172.21.15.97:0/655489830 >> v2:172.21.15.190:6809/239686547 conn(0xdf5240 0xea5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.030 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.038+0000 7fa11df9c640 1 --2- 172.21.15.97:0/655489830 >> v2:172.21.15.190:6809/239686547 conn(0xdf5240 0xea5a80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.043 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa11df9c640 1 -- 172.21.15.97:0/655489830 <== osd.2 v2:172.21.15.190:6809/239686547 1 ==== osd_op_reply(1 obj2 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fa11000a430 con 0xdf5240 2024-03-27T08:40:29.043 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 >> v2:172.21.15.190:6809/239686547 conn(0xdf5240 msgr2=0xea5a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> v2:172.21.15.190:6809/239686547 conn(0xdf5240 0xea5a80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa104037030 msgr2=0x7fa104039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa104037030 0x7fa104039540 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 msgr2=0xe9d390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe9d390 crc :-1 s=READY pgs=12951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 shutdown_connections 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> v2:172.21.15.190:6809/239686547 conn(0xdf5240 0xea5a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa104037030 0x7fa104039540 unknown :-1 s=CLOSED pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 --2- 172.21.15.97:0/655489830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0xdf48f0 0xe9d390 unknown :-1 s=CLOSED pgs=12951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 >> 172.21.15.97:0/655489830 conn(0xd28ea0 msgr2=0xe9e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 shutdown_connections 2024-03-27T08:40:29.044 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.051+0000 7fa125320ec0 1 -- 172.21.15.97:0/655489830 wait complete. 2024-03-27T08:40:29.045 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.045 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj3 /etc/passwd 2024-03-27T08:40:29.081 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.090+0000 7f7481ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.086 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.095+0000 7f7481ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.087 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.095+0000 7f7481ec9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.087 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.096+0000 7f7481ec9ec0 1 Processor -- start 2024-03-27T08:40:29.087 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.096+0000 7f7481ec9ec0 1 -- start start 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f7481ec9ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa62430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f7481ec9ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x9340b0 con 0x9b48f0 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f747a39b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa62430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f747a39b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa62430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46134/0 (socket says 172.21.15.97:46134) 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f747a39b640 1 -- 172.21.15.97:0/3690121294 learned_addr learned my addr 172.21.15.97:0/3690121294 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f747a39b640 1 -- 172.21.15.97:0/3690121294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x8d41e0 con 0x9b48f0 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f747a39b640 1 --2- 172.21.15.97:0/3690121294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa62430 crc :-1 s=READY pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860ae15b50790a57 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f7478b98640 1 -- 172.21.15.97:0/3690121294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7464029ad0 con 0x9b48f0 2024-03-27T08:40:29.089 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f7478b98640 1 -- 172.21.15.97:0/3690121294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f746400a040 con 0x9b48f0 2024-03-27T08:40:29.090 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.097+0000 7f7478b98640 1 -- 172.21.15.97:0/3690121294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f746400a1b0 con 0x9b48f0 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/3690121294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 msgr2=0xa62430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/3690121294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa62430 crc :-1 s=READY pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/3690121294 shutdown_connections 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/3690121294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa62430 unknown :-1 s=CLOSED pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/3690121294 >> 172.21.15.97:0/3690121294 conn(0x8e8ea0 msgr2=0xa5e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/3690121294 shutdown_connections 2024-03-27T08:40:29.091 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.099+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/3690121294 wait complete. 2024-03-27T08:40:29.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.100+0000 7f7481ec9ec0 1 Processor -- start 2024-03-27T08:40:29.092 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.100+0000 7f7481ec9ec0 1 -- start start 2024-03-27T08:40:29.093 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.101+0000 7f7481ec9ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa65e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.101+0000 7f7481ec9ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x931b30 con 0x9b48f0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.101+0000 7f747a39b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.101+0000 7f747a39b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa65e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46148/0 (socket says 172.21.15.97:46148) 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.101+0000 7f747a39b640 1 -- 172.21.15.97:0/2152606864 learned_addr learned my addr 172.21.15.97:0/2152606864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f747a39b640 1 -- 172.21.15.97:0/2152606864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x8c6e30 con 0x9b48f0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f747a39b640 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa65e80 crc :-1 s=READY pgs=12955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f7472ffd640 1 -- 172.21.15.97:0/2152606864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f746403b040 con 0x9b48f0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f7472ffd640 1 -- 172.21.15.97:0/2152606864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7464036ba0 con 0x9b48f0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x8c6890 con 0x9b48f0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f7472ffd640 1 -- 172.21.15.97:0/2152606864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f746400a1b0 con 0x9b48f0 2024-03-27T08:40:29.094 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.102+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0xa656e0 con 0x9b48f0 2024-03-27T08:40:29.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.103+0000 7f7472ffd640 1 -- 172.21.15.97:0/2152606864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f746403e040 con 0x9b48f0 2024-03-27T08:40:29.095 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.103+0000 7f7472ffd640 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7460036fe0 0x7f74600394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.103+0000 7f7472ffd640 1 -- 172.21.15.97:0/2152606864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f746402e040 con 0x9b48f0 2024-03-27T08:40:29.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.103+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> v2:172.21.15.97:6805/1668842645 conn(0x9b5240 0xaf93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.103+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 201.1 201:8dd16f86:::obj3:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0xa651e0 con 0x9b5240 2024-03-27T08:40:29.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.103+0000 7f7479b9a640 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7460036fe0 0x7f74600394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.104+0000 7f7479b9a640 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7460036fe0 0x7f74600394f0 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.096 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.104+0000 7f747ab9c640 1 --2- 172.21.15.97:0/2152606864 >> v2:172.21.15.97:6805/1668842645 conn(0x9b5240 0xaf93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.099 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.108+0000 7f747ab9c640 1 --2- 172.21.15.97:0/2152606864 >> v2:172.21.15.97:6805/1668842645 conn(0x9b5240 0xaf93e0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.119+0000 7f747ab9c640 1 -- 172.21.15.97:0/2152606864 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj3 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f746c00a430 con 0x9b5240 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.119+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 >> v2:172.21.15.97:6805/1668842645 conn(0x9b5240 msgr2=0xaf93e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.119+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> v2:172.21.15.97:6805/1668842645 conn(0x9b5240 0xaf93e0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7460036fe0 msgr2=0x7f74600394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7460036fe0 0x7f74600394f0 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 msgr2=0xa65e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa65e80 crc :-1 s=READY pgs=12955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 shutdown_connections 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> v2:172.21.15.97:6805/1668842645 conn(0x9b5240 0xaf93e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7460036fe0 0x7f74600394f0 unknown :-1 s=CLOSED pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.113 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 --2- 172.21.15.97:0/2152606864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x9b48f0 0xa65e80 unknown :-1 s=CLOSED pgs=12955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.113 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 >> 172.21.15.97:0/2152606864 conn(0x8e8ea0 msgr2=0xa5e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.113 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 shutdown_connections 2024-03-27T08:40:29.113 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.120+0000 7f7481ec9ec0 1 -- 172.21.15.97:0/2152606864 wait complete. 2024-03-27T08:40:29.114 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.114 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj4 /etc/passwd 2024-03-27T08:40:29.148 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.157+0000 7f260ab26ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.153 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.162+0000 7f260ab26ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.153 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.162+0000 7f260ab26ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.154 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.162+0000 7f260ab26ec0 1 Processor -- start 2024-03-27T08:40:29.154 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.162+0000 7f260ab26ec0 1 -- start start 2024-03-27T08:40:29.156 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.163+0000 7f260ab26ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x18801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.156 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.163+0000 7f260ab26ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x17560b0 con 0x17f0f70 2024-03-27T08:40:29.156 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.163+0000 7f2602f9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x18801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.156 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.163+0000 7f2602f9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x18801d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46170/0 (socket says 172.21.15.97:46170) 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.163+0000 7f2602f9b640 1 -- 172.21.15.97:0/2754835403 learned_addr learned my addr 172.21.15.97:0/2754835403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.164+0000 7f2602f9b640 1 -- 172.21.15.97:0/2754835403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x16f61e0 con 0x17f0f70 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.164+0000 7f2602f9b640 1 --2- 172.21.15.97:0/2754835403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x18801d0 crc :-1 s=READY pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd7dc736a369f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.164+0000 7f2601798640 1 -- 172.21.15.97:0/2754835403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f25ec007770 con 0x17f0f70 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.164+0000 7f2601798640 1 -- 172.21.15.97:0/2754835403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f25ec0078d0 con 0x17f0f70 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.164+0000 7f2601798640 1 -- 172.21.15.97:0/2754835403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f25ec007b70 con 0x17f0f70 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2754835403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 msgr2=0x18801d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2754835403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x18801d0 crc :-1 s=READY pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2754835403 shutdown_connections 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2754835403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x18801d0 unknown :-1 s=CLOSED pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2754835403 >> 172.21.15.97:0/2754835403 conn(0x170aea0 msgr2=0x17e9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2754835403 shutdown_connections 2024-03-27T08:40:29.157 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2754835403 wait complete. 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.165+0000 7f260ab26ec0 1 Processor -- start 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.166+0000 7f260ab26ec0 1 -- start start 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.166+0000 7f260ab26ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x17e9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.166+0000 7f260ab26ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1753b30 con 0x17f0f70 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f2602f9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x17e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f2602f9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x17e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46174/0 (socket says 172.21.15.97:46174) 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f2602f9b640 1 -- 172.21.15.97:0/2075415517 learned_addr learned my addr 172.21.15.97:0/2075415517 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f2602f9b640 1 -- 172.21.15.97:0/2075415517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x16e8e30 con 0x17f0f70 2024-03-27T08:40:29.159 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f2602f9b640 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x17e9730 crc :-1 s=READY pgs=12959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.160 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f25fb7fe640 1 -- 172.21.15.97:0/2075415517 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f25ec035750 con 0x17f0f70 2024-03-27T08:40:29.160 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f25fb7fe640 1 -- 172.21.15.97:0/2075415517 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f25ec0358b0 con 0x17f0f70 2024-03-27T08:40:29.160 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f25fb7fe640 1 -- 172.21.15.97:0/2075415517 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f25ec035b50 con 0x17f0f70 2024-03-27T08:40:29.160 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x16e8890 con 0x17f0f70 2024-03-27T08:40:29.160 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.167+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1908810 con 0x17f0f70 2024-03-27T08:40:29.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.168+0000 7f25fb7fe640 1 -- 172.21.15.97:0/2075415517 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f25ec035d00 con 0x17f0f70 2024-03-27T08:40:29.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.168+0000 7f25fb7fe640 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f25e8037030 0x7f25e8039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.168+0000 7f25fb7fe640 1 -- 172.21.15.97:0/2075415517 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f25ec03a040 con 0x17f0f70 2024-03-27T08:40:29.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.168+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> v2:172.21.15.190:6805/971408905 conn(0x1882360 0x1916910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.168+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 201.0 201:0ee9ae15:::obj4:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x1909150 con 0x1882360 2024-03-27T08:40:29.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.169+0000 7f260279a640 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f25e8037030 0x7f25e8039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.169+0000 7f260279a640 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f25e8037030 0x7f25e8039540 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.162 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.170+0000 7f260379c640 1 --2- 172.21.15.97:0/2075415517 >> v2:172.21.15.190:6805/971408905 conn(0x1882360 0x1916910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.165 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.174+0000 7f260379c640 1 --2- 172.21.15.97:0/2075415517 >> v2:172.21.15.190:6805/971408905 conn(0x1882360 0x1916910 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260379c640 1 -- 172.21.15.97:0/2075415517 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj4 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f25f400a430 con 0x1882360 2024-03-27T08:40:29.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 >> v2:172.21.15.190:6805/971408905 conn(0x1882360 msgr2=0x1916910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> v2:172.21.15.190:6805/971408905 conn(0x1882360 0x1916910 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f25e8037030 msgr2=0x7f25e8039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f25e8037030 0x7f25e8039540 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 msgr2=0x17e9730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x17e9730 crc :-1 s=READY pgs=12959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 shutdown_connections 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> v2:172.21.15.190:6805/971408905 conn(0x1882360 0x1916910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f25e8037030 0x7f25e8039540 unknown :-1 s=CLOSED pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 --2- 172.21.15.97:0/2075415517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x17f0f70 0x17e9730 unknown :-1 s=CLOSED pgs=12959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 >> 172.21.15.97:0/2075415517 conn(0x170aea0 msgr2=0x187fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 shutdown_connections 2024-03-27T08:40:29.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.187+0000 7f260ab26ec0 1 -- 172.21.15.97:0/2075415517 wait complete. 2024-03-27T08:40:29.181 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.181 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj5 /etc/passwd 2024-03-27T08:40:29.215 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.224+0000 7f7d25440ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.220 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.229+0000 7f7d25440ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.220 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.229+0000 7f7d25440ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.221 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.229+0000 7f7d25440ec0 1 Processor -- start 2024-03-27T08:40:29.221 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.229+0000 7f7d25440ec0 1 -- start start 2024-03-27T08:40:29.223 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.230+0000 7f7d25440ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x15c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.223 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.230+0000 7f7d25440ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x140b0b0 con 0x148b8f0 2024-03-27T08:40:29.223 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d1d59b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x15c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d1d59b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x15c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46186/0 (socket says 172.21.15.97:46186) 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d1d59b640 1 -- 172.21.15.97:0/2263809811 learned_addr learned my addr 172.21.15.97:0/2263809811 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d1d59b640 1 -- 172.21.15.97:0/2263809811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x13ab1e0 con 0x148b8f0 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d1d59b640 1 --2- 172.21.15.97:0/2263809811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x15c8a30 crc :-1 s=READY pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ae721530d063e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d16ffd640 1 -- 172.21.15.97:0/2263809811 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d1000c490 con 0x148b8f0 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d16ffd640 1 -- 172.21.15.97:0/2263809811 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7d1000c5f0 con 0x148b8f0 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.231+0000 7f7d16ffd640 1 -- 172.21.15.97:0/2263809811 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d1000c890 con 0x148b8f0 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2263809811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 msgr2=0x15c8a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2263809811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x15c8a30 crc :-1 s=READY pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2263809811 shutdown_connections 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2263809811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x15c8a30 unknown :-1 s=CLOSED pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2263809811 >> 172.21.15.97:0/2263809811 conn(0x13bfea0 msgr2=0x1539080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.224 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2263809811 shutdown_connections 2024-03-27T08:40:29.225 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2263809811 wait complete. 2024-03-27T08:40:29.228 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.232+0000 7f7d25440ec0 1 Processor -- start 2024-03-27T08:40:29.228 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.233+0000 7f7d25440ec0 1 -- start start 2024-03-27T08:40:29.228 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.233+0000 7f7d25440ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x149eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.228 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.233+0000 7f7d25440ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1408b30 con 0x148b8f0 2024-03-27T08:40:29.228 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d1d59b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x149eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d1d59b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x149eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46190/0 (socket says 172.21.15.97:46190) 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d1d59b640 1 -- 172.21.15.97:0/2255840886 learned_addr learned my addr 172.21.15.97:0/2255840886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d1d59b640 1 -- 172.21.15.97:0/2255840886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x139de30 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d1d59b640 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x149eca0 crc :-1 s=READY pgs=12963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d15ffb640 1 -- 172.21.15.97:0/2255840886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d100111e0 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d15ffb640 1 -- 172.21.15.97:0/2255840886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7d100119d0 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x139d890 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d15ffb640 1 -- 172.21.15.97:0/2255840886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d10011b40 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.234+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x149e850 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.235+0000 7f7d15ffb640 1 -- 172.21.15.97:0/2255840886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7d10011d40 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.235+0000 7f7d15ffb640 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d04036fe0 0x7f7d040394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.235+0000 7f7d15ffb640 1 -- 172.21.15.97:0/2255840886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f7d10048b80 con 0x148b8f0 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.235+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> v2:172.21.15.97:6801/3203341735 conn(0x148c410 0x15cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.235+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 --> v2:172.21.15.97:6801/3203341735 -- osd_op(unknown.0.0:1 201.3 201:c52c9666:::obj5:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x15d0270 con 0x148c410 2024-03-27T08:40:29.229 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.235+0000 7f7d1cd9a640 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d04036fe0 0x7f7d040394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.230 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.236+0000 7f7d1dd9c640 1 --2- 172.21.15.97:0/2255840886 >> v2:172.21.15.97:6801/3203341735 conn(0x148c410 0x15cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.230 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.236+0000 7f7d1cd9a640 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d04036fe0 0x7f7d040394f0 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.230 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.239+0000 7f7d1dd9c640 1 --2- 172.21.15.97:0/2255840886 >> v2:172.21.15.97:6801/3203341735 conn(0x148c410 0x15cfd30 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.243 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d1dd9c640 1 -- 172.21.15.97:0/2255840886 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== osd_op_reply(1 obj5 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7d0800a430 con 0x148c410 2024-03-27T08:40:29.243 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 >> v2:172.21.15.97:6801/3203341735 conn(0x148c410 msgr2=0x15cfd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.243 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> v2:172.21.15.97:6801/3203341735 conn(0x148c410 0x15cfd30 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d04036fe0 msgr2=0x7f7d040394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d04036fe0 0x7f7d040394f0 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 msgr2=0x149eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x149eca0 crc :-1 s=READY pgs=12963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 shutdown_connections 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> v2:172.21.15.97:6801/3203341735 conn(0x148c410 0x15cfd30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d04036fe0 0x7f7d040394f0 unknown :-1 s=CLOSED pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 --2- 172.21.15.97:0/2255840886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x148b8f0 0x149eca0 unknown :-1 s=CLOSED pgs=12963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 >> 172.21.15.97:0/2255840886 conn(0x13bfea0 msgr2=0x1539080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 shutdown_connections 2024-03-27T08:40:29.244 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.250+0000 7f7d25440ec0 1 -- 172.21.15.97:0/2255840886 wait complete. 2024-03-27T08:40:29.245 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.245 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj6 /etc/passwd 2024-03-27T08:40:29.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.288+0000 7f161a06eec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.285 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.293+0000 7f161a06eec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.285 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.293+0000 7f161a06eec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.285 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.294+0000 7f161a06eec0 1 Processor -- start 2024-03-27T08:40:29.285 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.294+0000 7f161a06eec0 1 -- start start 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.295+0000 7f161a06eec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x15e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.295+0000 7f161a06eec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x14b50b0 con 0x1536000 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.295+0000 7f161219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x15e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.295+0000 7f161219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x15e1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46206/0 (socket says 172.21.15.97:46206) 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.295+0000 7f161219b640 1 -- 172.21.15.97:0/2736963889 learned_addr learned my addr 172.21.15.97:0/2736963889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.296+0000 7f161219b640 1 -- 172.21.15.97:0/2736963889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x14551e0 con 0x1536000 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.296+0000 7f161219b640 1 --2- 172.21.15.97:0/2736963889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x15e1940 crc :-1 s=READY pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54db22c0aa21eac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.302 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.296+0000 7f1610998640 1 -- 172.21.15.97:0/2736963889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f15fc02a9f0 con 0x1536000 2024-03-27T08:40:29.303 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.296+0000 7f1610998640 1 -- 172.21.15.97:0/2736963889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f15fc02ab50 con 0x1536000 2024-03-27T08:40:29.303 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.296+0000 7f1610998640 1 -- 172.21.15.97:0/2736963889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f15fc02adf0 con 0x1536000 2024-03-27T08:40:29.303 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 -- 172.21.15.97:0/2736963889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 msgr2=0x15e1940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.303 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 --2- 172.21.15.97:0/2736963889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x15e1940 crc :-1 s=READY pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.303 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 -- 172.21.15.97:0/2736963889 shutdown_connections 2024-03-27T08:40:29.303 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 --2- 172.21.15.97:0/2736963889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x15e1940 unknown :-1 s=CLOSED pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.305 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 -- 172.21.15.97:0/2736963889 >> 172.21.15.97:0/2736963889 conn(0x1469ea0 msgr2=0x15e6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.305 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 -- 172.21.15.97:0/2736963889 shutdown_connections 2024-03-27T08:40:29.305 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.297+0000 7f161a06eec0 1 -- 172.21.15.97:0/2736963889 wait complete. 2024-03-27T08:40:29.305 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.298+0000 7f161a06eec0 1 Processor -- start 2024-03-27T08:40:29.305 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.298+0000 7f161a06eec0 1 -- start start 2024-03-27T08:40:29.305 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.299+0000 7f161a06eec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x16781c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.299+0000 7f161a06eec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x14b2b30 con 0x1536000 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.299+0000 7f161219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x16781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.299+0000 7f161219b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x16781c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46210/0 (socket says 172.21.15.97:46210) 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.299+0000 7f161219b640 1 -- 172.21.15.97:0/1873373721 learned_addr learned my addr 172.21.15.97:0/1873373721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f161219b640 1 -- 172.21.15.97:0/1873373721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1447e30 con 0x1536000 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f161219b640 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x16781c0 crc :-1 s=READY pgs=12967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.306 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f160affd640 1 -- 172.21.15.97:0/1873373721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f15fc02a970 con 0x1536000 2024-03-27T08:40:29.307 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f160affd640 1 -- 172.21.15.97:0/1873373721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f15fc0073f0 con 0x1536000 2024-03-27T08:40:29.307 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1447890 con 0x1536000 2024-03-27T08:40:29.307 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f160affd640 1 -- 172.21.15.97:0/1873373721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f15fc007690 con 0x1536000 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.300+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x16708a0 con 0x1536000 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.301+0000 7f160affd640 1 -- 172.21.15.97:0/1873373721 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f15fc032040 con 0x1536000 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.301+0000 7f160affd640 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f15f8036fe0 0x7f15f80394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.301+0000 7f160affd640 1 -- 172.21.15.97:0/1873373721 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f15fc03b040 con 0x1536000 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.301+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> v2:172.21.15.97:6805/1668842645 conn(0x1550060 0x15e1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.301+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 201.4 201:233a42c1:::obj6:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x15cca10 con 0x1550060 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.301+0000 7f161199a640 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f15f8036fe0 0x7f15f80394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.302+0000 7f161199a640 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f15f8036fe0 0x7f15f80394f0 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.302+0000 7f161299c640 1 --2- 172.21.15.97:0/1873373721 >> v2:172.21.15.97:6805/1668842645 conn(0x1550060 0x15e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.306+0000 7f161299c640 1 --2- 172.21.15.97:0/1873373721 >> v2:172.21.15.97:6805/1668842645 conn(0x1550060 0x15e1460 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161299c640 1 -- 172.21.15.97:0/1873373721 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj6 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f160400a430 con 0x1550060 2024-03-27T08:40:29.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 >> v2:172.21.15.97:6805/1668842645 conn(0x1550060 msgr2=0x15e1460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> v2:172.21.15.97:6805/1668842645 conn(0x1550060 0x15e1460 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f15f8036fe0 msgr2=0x7f15f80394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f15f8036fe0 0x7f15f80394f0 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 msgr2=0x16781c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x16781c0 crc :-1 s=READY pgs=12967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 shutdown_connections 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> v2:172.21.15.97:6805/1668842645 conn(0x1550060 0x15e1460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f15f8036fe0 0x7f15f80394f0 unknown :-1 s=CLOSED pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 --2- 172.21.15.97:0/1873373721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1536000 0x16781c0 unknown :-1 s=CLOSED pgs=12967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 >> 172.21.15.97:0/1873373721 conn(0x1469ea0 msgr2=0x15e6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 shutdown_connections 2024-03-27T08:40:29.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.318+0000 7f161a06eec0 1 -- 172.21.15.97:0/1873373721 wait complete. 2024-03-27T08:40:29.314 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.314 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj7 /etc/passwd 2024-03-27T08:40:29.346 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.355+0000 7f6594684ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.353 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.360+0000 7f6594684ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.353 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.360+0000 7f6594684ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.353 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.361+0000 7f6594684ec0 1 Processor -- start 2024-03-27T08:40:29.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.361+0000 7f6594684ec0 1 -- start start 2024-03-27T08:40:29.355 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f6594684ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x9076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f6594684ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7dd0b0 con 0x85d8f0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f65877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x9076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f65877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x9076d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46224/0 (socket says 172.21.15.97:46224) 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f65877fe640 1 -- 172.21.15.97:0/412670041 learned_addr learned my addr 172.21.15.97:0/412670041 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f65877fe640 1 -- 172.21.15.97:0/412670041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x77d1e0 con 0x85d8f0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.362+0000 7f65877fe640 1 --2- 172.21.15.97:0/412670041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x9076d0 crc :-1 s=READY pgs=12970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258e582a3243a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6585ffb640 1 -- 172.21.15.97:0/412670041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f657802e040 con 0x85d8f0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6585ffb640 1 -- 172.21.15.97:0/412670041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6578032300 con 0x85d8f0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6585ffb640 1 -- 172.21.15.97:0/412670041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f65780325a0 con 0x85d8f0 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 -- 172.21.15.97:0/412670041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 msgr2=0x9076d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.356 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 --2- 172.21.15.97:0/412670041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x9076d0 crc :-1 s=READY pgs=12970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 -- 172.21.15.97:0/412670041 shutdown_connections 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 --2- 172.21.15.97:0/412670041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x9076d0 unknown :-1 s=CLOSED pgs=12970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 -- 172.21.15.97:0/412670041 >> 172.21.15.97:0/412670041 conn(0x791ea0 msgr2=0x9086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 -- 172.21.15.97:0/412670041 shutdown_connections 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.363+0000 7f6594684ec0 1 -- 172.21.15.97:0/412670041 wait complete. 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.364+0000 7f6594684ec0 1 Processor -- start 2024-03-27T08:40:29.357 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.364+0000 7f6594684ec0 1 -- start start 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.365+0000 7f6594684ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x906ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.365+0000 7f6594684ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7dab30 con 0x85d8f0 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.365+0000 7f65877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x906ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.365+0000 7f65877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x906ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46230/0 (socket says 172.21.15.97:46230) 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.365+0000 7f65877fe640 1 -- 172.21.15.97:0/3675564428 learned_addr learned my addr 172.21.15.97:0/3675564428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.365+0000 7f65877fe640 1 -- 172.21.15.97:0/3675564428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x76fe30 con 0x85d8f0 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.366+0000 7f65877fe640 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x906ed0 crc :-1 s=READY pgs=12971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.362 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.366+0000 7f6584ff9640 1 -- 172.21.15.97:0/3675564428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6578032dc0 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.366+0000 7f6584ff9640 1 -- 172.21.15.97:0/3675564428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6578035450 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.366+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x76f890 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.366+0000 7f6584ff9640 1 -- 172.21.15.97:0/3675564428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f65780356f0 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.366+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x9083a0 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.367+0000 7f6584ff9640 1 -- 172.21.15.97:0/3675564428 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f65780358a0 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.367+0000 7f6584ff9640 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6574037030 0x7f6574039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.367+0000 7f6584ff9640 1 -- 172.21.15.97:0/3675564428 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f657803a040 con 0x85d8f0 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.367+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> v2:172.21.15.97:6805/1668842645 conn(0x85e240 0x8f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.367+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 201.9 201:9a09113a:::obj7:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x8f6430 con 0x85e240 2024-03-27T08:40:29.363 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.367+0000 7f6586ffd640 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6574037030 0x7f6574039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.364 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.368+0000 7f658cf9c640 1 --2- 172.21.15.97:0/3675564428 >> v2:172.21.15.97:6805/1668842645 conn(0x85e240 0x8f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.364 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.368+0000 7f6586ffd640 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6574037030 0x7f6574039540 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.364 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.370+0000 7f658cf9c640 1 --2- 172.21.15.97:0/3675564428 >> v2:172.21.15.97:6805/1668842645 conn(0x85e240 0x8f3590 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f658cf9c640 1 -- 172.21.15.97:0/3675564428 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj7 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f658000c430 con 0x85e240 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 >> v2:172.21.15.97:6805/1668842645 conn(0x85e240 msgr2=0x8f3590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> v2:172.21.15.97:6805/1668842645 conn(0x85e240 0x8f3590 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6574037030 msgr2=0x7f6574039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6574037030 0x7f6574039540 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 msgr2=0x906ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.375 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x906ed0 crc :-1 s=READY pgs=12971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 shutdown_connections 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> v2:172.21.15.97:6805/1668842645 conn(0x85e240 0x8f3590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f6574037030 0x7f6574039540 unknown :-1 s=CLOSED pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 --2- 172.21.15.97:0/3675564428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x85d8f0 0x906ed0 unknown :-1 s=CLOSED pgs=12971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 >> 172.21.15.97:0/3675564428 conn(0x791ea0 msgr2=0x9086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 shutdown_connections 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.382+0000 7f6594684ec0 1 -- 172.21.15.97:0/3675564428 wait complete. 2024-03-27T08:40:29.376 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.377 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj8 /etc/passwd 2024-03-27T08:40:29.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.419+0000 7f738502dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.416 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.424+0000 7f738502dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.416 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.424+0000 7f738502dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.416 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.425+0000 7f738502dec0 1 Processor -- start 2024-03-27T08:40:29.416 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.425+0000 7f738502dec0 1 -- start start 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f738502dec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a4ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f738502dec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x191f0b0 con 0x199f8f0 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f737d19b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a4ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f737d19b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a4ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46256/0 (socket says 172.21.15.97:46256) 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f737d19b640 1 -- 172.21.15.97:0/1100998048 learned_addr learned my addr 172.21.15.97:0/1100998048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f737d19b640 1 -- 172.21.15.97:0/1100998048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18bf1e0 con 0x199f8f0 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f737d19b640 1 --2- 172.21.15.97:0/1100998048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a4ebf0 crc :-1 s=READY pgs=12974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609b940b89bc592a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f7376ffd640 1 -- 172.21.15.97:0/1100998048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f73680061e0 con 0x199f8f0 2024-03-27T08:40:29.423 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f7376ffd640 1 -- 172.21.15.97:0/1100998048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f73680019c0 con 0x199f8f0 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.427+0000 7f7376ffd640 1 -- 172.21.15.97:0/1100998048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7368006340 con 0x199f8f0 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 -- 172.21.15.97:0/1100998048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 msgr2=0x1a4ebf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 --2- 172.21.15.97:0/1100998048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a4ebf0 crc :-1 s=READY pgs=12974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 -- 172.21.15.97:0/1100998048 shutdown_connections 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 --2- 172.21.15.97:0/1100998048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a4ebf0 unknown :-1 s=CLOSED pgs=12974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 -- 172.21.15.97:0/1100998048 >> 172.21.15.97:0/1100998048 conn(0x18d3ea0 msgr2=0x19b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 -- 172.21.15.97:0/1100998048 shutdown_connections 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.428+0000 7f738502dec0 1 -- 172.21.15.97:0/1100998048 wait complete. 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.429+0000 7f738502dec0 1 Processor -- start 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.429+0000 7f738502dec0 1 -- start start 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.430+0000 7f738502dec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.430+0000 7f738502dec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x191cb30 con 0x199f8f0 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.430+0000 7f737d19b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.424 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.430+0000 7f737d19b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46264/0 (socket says 172.21.15.97:46264) 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.430+0000 7f737d19b640 1 -- 172.21.15.97:0/2709073656 learned_addr learned my addr 172.21.15.97:0/2709073656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f737d19b640 1 -- 172.21.15.97:0/2709073656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18b1e30 con 0x199f8f0 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f737d19b640 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a365e0 crc :-1 s=READY pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f7375ffb640 1 -- 172.21.15.97:0/2709073656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7368004b90 con 0x199f8f0 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f7375ffb640 1 -- 172.21.15.97:0/2709073656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7368004cf0 con 0x199f8f0 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f7375ffb640 1 -- 172.21.15.97:0/2709073656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7368012330 con 0x199f8f0 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x18b1890 con 0x199f8f0 2024-03-27T08:40:29.425 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.431+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1a39230 con 0x199f8f0 2024-03-27T08:40:29.426 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.432+0000 7f7375ffb640 1 -- 172.21.15.97:0/2709073656 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7368012520 con 0x199f8f0 2024-03-27T08:40:29.426 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.432+0000 7f7375ffb640 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7364037030 0x7f7364039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.426 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.432+0000 7f7375ffb640 1 -- 172.21.15.97:0/2709073656 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f73680486b0 con 0x199f8f0 2024-03-27T08:40:29.426 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.433+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> v2:172.21.15.97:6805/1668842645 conn(0x19a0410 0x1ae4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.426 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.433+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 201.5 201:a557efb1:::obj8:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x1a3a0b0 con 0x19a0410 2024-03-27T08:40:29.427 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.433+0000 7f737c99a640 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7364037030 0x7f7364039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.427 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.434+0000 7f737c99a640 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7364037030 0x7f7364039540 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.427 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.434+0000 7f737d99c640 1 --2- 172.21.15.97:0/2709073656 >> v2:172.21.15.97:6805/1668842645 conn(0x19a0410 0x1ae4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.429 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.437+0000 7f737d99c640 1 --2- 172.21.15.97:0/2709073656 >> v2:172.21.15.97:6805/1668842645 conn(0x19a0410 0x1ae4190 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.441 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f737d99c640 1 -- 172.21.15.97:0/2709073656 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj8 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f737000c430 con 0x19a0410 2024-03-27T08:40:29.441 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 >> v2:172.21.15.97:6805/1668842645 conn(0x19a0410 msgr2=0x1ae4190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.441 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> v2:172.21.15.97:6805/1668842645 conn(0x19a0410 0x1ae4190 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.441 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7364037030 msgr2=0x7f7364039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7364037030 0x7f7364039540 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 msgr2=0x1a365e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a365e0 crc :-1 s=READY pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 shutdown_connections 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> v2:172.21.15.97:6805/1668842645 conn(0x19a0410 0x1ae4190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7364037030 0x7f7364039540 unknown :-1 s=CLOSED pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 --2- 172.21.15.97:0/2709073656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x199f8f0 0x1a365e0 unknown :-1 s=CLOSED pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 >> 172.21.15.97:0/2709073656 conn(0x18d3ea0 msgr2=0x19bbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 shutdown_connections 2024-03-27T08:40:29.442 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.449+0000 7f738502dec0 1 -- 172.21.15.97:0/2709073656 wait complete. 2024-03-27T08:40:29.443 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.443 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj9 /etc/passwd 2024-03-27T08:40:29.479 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.488+0000 7f7dbe630ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.484 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.493+0000 7f7dbe630ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.484 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.493+0000 7f7dbe630ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.485 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.494+0000 7f7dbe630ec0 1 Processor -- start 2024-03-27T08:40:29.485 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.494+0000 7f7dbe630ec0 1 -- start start 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7dbe630ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x1508630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7dbe630ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x13d70b0 con 0x1471f70 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db679b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x1508630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db679b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x1508630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46294/0 (socket says 172.21.15.97:46294) 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db679b640 1 -- 172.21.15.97:0/1524998591 learned_addr learned my addr 172.21.15.97:0/1524998591 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db679b640 1 -- 172.21.15.97:0/1524998591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x13771e0 con 0x1471f70 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db679b640 1 --2- 172.21.15.97:0/1524998591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x1508630 crc :-1 s=READY pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3a0034c2e19810 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db4f98640 1 -- 172.21.15.97:0/1524998591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7da0006380 con 0x1471f70 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db4f98640 1 -- 172.21.15.97:0/1524998591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7da00064e0 con 0x1471f70 2024-03-27T08:40:29.487 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.495+0000 7f7db4f98640 1 -- 172.21.15.97:0/1524998591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7da00067e0 con 0x1471f70 2024-03-27T08:40:29.488 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/1524998591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 msgr2=0x1508630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.488 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/1524998591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x1508630 crc :-1 s=READY pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.488 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/1524998591 shutdown_connections 2024-03-27T08:40:29.488 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/1524998591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x1508630 unknown :-1 s=CLOSED pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.489 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/1524998591 >> 172.21.15.97:0/1524998591 conn(0x138bea0 msgr2=0x146b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.489 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/1524998591 shutdown_connections 2024-03-27T08:40:29.489 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.496+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/1524998591 wait complete. 2024-03-27T08:40:29.489 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.497+0000 7f7dbe630ec0 1 Processor -- start 2024-03-27T08:40:29.489 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.497+0000 7f7dbe630ec0 1 -- start start 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7dbe630ec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x14f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7dbe630ec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x13d4b30 con 0x1471f70 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7db679b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x14f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7db679b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x14f0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46310/0 (socket says 172.21.15.97:46310) 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7db679b640 1 -- 172.21.15.97:0/308106952 learned_addr learned my addr 172.21.15.97:0/308106952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7db679b640 1 -- 172.21.15.97:0/308106952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1369e30 con 0x1471f70 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7db679b640 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x14f0f90 crc :-1 s=READY pgs=12979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7daeffd640 1 -- 172.21.15.97:0/308106952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7da003a040 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7daeffd640 1 -- 172.21.15.97:0/308106952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7da0006e00 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7daeffd640 1 -- 172.21.15.97:0/308106952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7da0035580 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1369890 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.498+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1504970 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.499+0000 7f7daeffd640 1 -- 172.21.15.97:0/308106952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7da003d040 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.499+0000 7f7daeffd640 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d9c036fe0 0x7f7d9c0394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.499+0000 7f7daeffd640 1 -- 172.21.15.97:0/308106952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f7da002e040 con 0x1471f70 2024-03-27T08:40:29.492 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.500+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> v2:172.21.15.190:6805/971408905 conn(0x14728c0 0x14f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.495 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.500+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 --> v2:172.21.15.190:6805/971408905 -- osd_op(unknown.0.0:1 201.7 201:fe351e27:::obj9:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x15031f0 con 0x14728c0 2024-03-27T08:40:29.495 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.500+0000 7f7db5f9a640 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d9c036fe0 0x7f7d9c0394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.500+0000 7f7db5f9a640 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d9c036fe0 0x7f7d9c0394f0 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.501+0000 7f7db6f9c640 1 --2- 172.21.15.97:0/308106952 >> v2:172.21.15.190:6805/971408905 conn(0x14728c0 0x14f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.505+0000 7f7db6f9c640 1 --2- 172.21.15.97:0/308106952 >> v2:172.21.15.190:6805/971408905 conn(0x14728c0 0x14f19e0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.517+0000 7f7db6f9c640 1 -- 172.21.15.97:0/308106952 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== osd_op_reply(1 obj9 [writefull 0~11812] v706'2 uv2 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7da800a430 con 0x14728c0 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.517+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 >> v2:172.21.15.190:6805/971408905 conn(0x14728c0 msgr2=0x14f19e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.517+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> v2:172.21.15.190:6805/971408905 conn(0x14728c0 0x14f19e0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d9c036fe0 msgr2=0x7f7d9c0394f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d9c036fe0 0x7f7d9c0394f0 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 msgr2=0x14f0f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x14f0f90 crc :-1 s=READY pgs=12979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 shutdown_connections 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> v2:172.21.15.190:6805/971408905 conn(0x14728c0 0x14f19e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.510 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7d9c036fe0 0x7f7d9c0394f0 unknown :-1 s=CLOSED pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.511 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 --2- 172.21.15.97:0/308106952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x1471f70 0x14f0f90 unknown :-1 s=CLOSED pgs=12979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.511 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 >> 172.21.15.97:0/308106952 conn(0x138bea0 msgr2=0x1506d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.511 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 shutdown_connections 2024-03-27T08:40:29.511 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.518+0000 7f7dbe630ec0 1 -- 172.21.15.97:0/308106952 wait complete. 2024-03-27T08:40:29.511 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-27T08:40:29.511 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 7165a426-c096-42b4-86ea-e81ff2d1e62a put obj10 /etc/passwd 2024-03-27T08:40:29.545 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.554+0000 7f075882fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.550 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.559+0000 7f075882fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.550 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.559+0000 7f075882fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:29.551 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.560+0000 7f075882fec0 1 Processor -- start 2024-03-27T08:40:29.551 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.560+0000 7f075882fec0 1 -- start start 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075882fec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x190cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075882fec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x18770b0 con 0x190dc90 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075099b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x190cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075099b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x190cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46332/0 (socket says 172.21.15.97:46332) 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075099b640 1 -- 172.21.15.97:0/942316051 learned_addr learned my addr 172.21.15.97:0/942316051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075099b640 1 -- 172.21.15.97:0/942316051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18171e0 con 0x190dc90 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f075099b640 1 --2- 172.21.15.97:0/942316051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x190cf10 crc :-1 s=READY pgs=12982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c8fde69a09b58c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f074a7fc640 1 -- 172.21.15.97:0/942316051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f073c0061e0 con 0x190dc90 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f074a7fc640 1 -- 172.21.15.97:0/942316051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f073c006340 con 0x190dc90 2024-03-27T08:40:29.553 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.561+0000 7f074a7fc640 1 -- 172.21.15.97:0/942316051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f073c001b40 con 0x190dc90 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 -- 172.21.15.97:0/942316051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 msgr2=0x190cf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 --2- 172.21.15.97:0/942316051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x190cf10 crc :-1 s=READY pgs=12982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 -- 172.21.15.97:0/942316051 shutdown_connections 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 --2- 172.21.15.97:0/942316051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x190cf10 unknown :-1 s=CLOSED pgs=12982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 -- 172.21.15.97:0/942316051 >> 172.21.15.97:0/942316051 conn(0x182bea0 msgr2=0x19a3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 -- 172.21.15.97:0/942316051 shutdown_connections 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.562+0000 7f075882fec0 1 -- 172.21.15.97:0/942316051 wait complete. 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.563+0000 7f075882fec0 1 Processor -- start 2024-03-27T08:40:29.555 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.563+0000 7f075882fec0 1 -- start start 2024-03-27T08:40:29.558 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075882fec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x1a3e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.558 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075882fec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x1874b30 con 0x190dc90 2024-03-27T08:40:29.558 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075099b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x1a3e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.558 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075099b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x1a3e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46346/0 (socket says 172.21.15.97:46346) 2024-03-27T08:40:29.558 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075099b640 1 -- 172.21.15.97:0/1652173302 learned_addr learned my addr 172.21.15.97:0/1652173302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075099b640 1 -- 172.21.15.97:0/1652173302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1809e30 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075099b640 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x1a3e200 crc :-1 s=READY pgs=12983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f07497fa640 1 -- 172.21.15.97:0/1652173302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f073c0049a0 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f07497fa640 1 -- 172.21.15.97:0/1652173302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f073c004b00 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1809890 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f07497fa640 1 -- 172.21.15.97:0/1652173302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f073c004da0 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.564+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1a3ddb0 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.565+0000 7f07497fa640 1 -- 172.21.15.97:0/1652173302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f073c014410 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.565+0000 7f07497fa640 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f072c037030 0x7f072c039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.565+0000 7f07497fa640 1 -- 172.21.15.97:0/1652173302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(706..706 src has 399..706) ==== 4179+0+0 (crc 0 0 0) 0x7f073c048670 con 0x190dc90 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.566+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> v2:172.21.15.97:6805/1668842645 conn(0x190f880 0x19098b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.566+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 --> v2:172.21.15.97:6805/1668842645 -- osd_op(unknown.0.0:1 201.b 201:d1337354:::obj10:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e706) -- 0x1909df0 con 0x190f880 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.566+0000 7f074b7fe640 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f072c037030 0x7f072c039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:29.559 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.566+0000 7f074b7fe640 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f072c037030 0x7f072c039540 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.560 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.567+0000 7f075119c640 1 --2- 172.21.15.97:0/1652173302 >> v2:172.21.15.97:6805/1668842645 conn(0x190f880 0x19098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:29.561 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.570+0000 7f075119c640 1 --2- 172.21.15.97:0/1652173302 >> v2:172.21.15.97:6805/1668842645 conn(0x190f880 0x19098b0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:29.574 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.581+0000 7f075119c640 1 -- 172.21.15.97:0/1652173302 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== osd_op_reply(1 obj10 [writefull 0~11812] v706'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f074400c430 con 0x190f880 2024-03-27T08:40:29.574 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 >> v2:172.21.15.97:6805/1668842645 conn(0x190f880 msgr2=0x19098b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.574 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> v2:172.21.15.97:6805/1668842645 conn(0x190f880 0x19098b0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.574 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f072c037030 msgr2=0x7f072c039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f072c037030 0x7f072c039540 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 msgr2=0x1a3e200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x1a3e200 crc :-1 s=READY pgs=12983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 shutdown_connections 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> v2:172.21.15.97:6805/1668842645 conn(0x190f880 0x19098b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f072c037030 0x7f072c039540 unknown :-1 s=CLOSED pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 --2- 172.21.15.97:0/1652173302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x190dc90 0x1a3e200 unknown :-1 s=CLOSED pgs=12983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 >> 172.21.15.97:0/1652173302 conn(0x182bea0 msgr2=0x1910e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 shutdown_connections 2024-03-27T08:40:29.575 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:29.582+0000 7f075882fec0 1 -- 172.21.15.97:0/1652173302 wait complete. 2024-03-27T08:40:29.576 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-27T08:40:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p 97a8ed02-de19-41fe-985b-30a778c6d918 put threemore /etc/passwd 2024-03-27T08:40:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.629+0000 7ffad163cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.634+0000 7ffad163cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.634+0000 7ffad163cec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.635+0000 7ffad163cec0 1 Processor -- start 2024-03-27T08:40:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.635+0000 7ffad163cec0 1 -- start start 2024-03-27T08:40:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.636+0000 7ffad163cec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2371ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.636+0000 7ffad163cec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x22450b0 con 0x22dbc90 2024-03-27T08:40:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.636+0000 7ffac979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2371ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.636+0000 7ffac979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2371ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40580/0 (socket says 172.21.15.97:40580) 2024-03-27T08:40:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.636+0000 7ffac979b640 1 -- 172.21.15.97:0/3615067469 learned_addr learned my addr 172.21.15.97:0/3615067469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.636+0000 7ffac979b640 1 -- 172.21.15.97:0/3615067469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x21e51e0 con 0x22dbc90 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffac979b640 1 --2- 172.21.15.97:0/3615067469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2371ee0 crc :-1 s=READY pgs=12986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666434895fdafbdf server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffac2ffd640 1 -- 172.21.15.97:0/3615067469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffab40060b0 con 0x22dbc90 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffac2ffd640 1 -- 172.21.15.97:0/3615067469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ffab4006210 con 0x22dbc90 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffac2ffd640 1 -- 172.21.15.97:0/3615067469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffab4001b10 con 0x22dbc90 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffad163cec0 1 -- 172.21.15.97:0/3615067469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 msgr2=0x2371ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffad163cec0 1 --2- 172.21.15.97:0/3615067469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2371ee0 crc :-1 s=READY pgs=12986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffad163cec0 1 -- 172.21.15.97:0/3615067469 shutdown_connections 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffad163cec0 1 --2- 172.21.15.97:0/3615067469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2371ee0 unknown :-1 s=CLOSED pgs=12986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffad163cec0 1 -- 172.21.15.97:0/3615067469 >> 172.21.15.97:0/3615067469 conn(0x21f9ea0 msgr2=0x2376040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:59.632 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.637+0000 7ffad163cec0 1 -- 172.21.15.97:0/3615067469 shutdown_connections 2024-03-27T08:40:59.632 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.638+0000 7ffad163cec0 1 -- 172.21.15.97:0/3615067469 wait complete. 2024-03-27T08:40:59.632 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.638+0000 7ffad163cec0 1 Processor -- start 2024-03-27T08:40:59.632 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.638+0000 7ffad163cec0 1 -- start start 2024-03-27T08:40:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.639+0000 7ffad163cec0 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2361bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.639+0000 7ffad163cec0 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x2242b30 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.639+0000 7ffac979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2361bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.639+0000 7ffac979b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2361bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40584/0 (socket says 172.21.15.97:40584) 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.639+0000 7ffac979b640 1 -- 172.21.15.97:0/1343649856 learned_addr learned my addr 172.21.15.97:0/1343649856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffac979b640 1 -- 172.21.15.97:0/1343649856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x21d7e30 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffac979b640 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2361bb0 crc :-1 s=READY pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffac1ffb640 1 -- 172.21.15.97:0/1343649856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffab4004b20 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffac1ffb640 1 -- 172.21.15.97:0/1343649856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ffab4004c80 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x21d7890 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffac1ffb640 1 -- 172.21.15.97:0/1343649856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ffab4012330 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.640+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2365390 con 0x22dbc90 2024-03-27T08:40:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.641+0000 7ffac1ffb640 1 -- 172.21.15.97:0/1343649856 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7ffab40124e0 con 0x22dbc90 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.641+0000 7ffac1ffb640 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffab0037030 0x7ffab0039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.641+0000 7ffac1ffb640 1 -- 172.21.15.97:0/1343649856 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(708..708 src has 399..708) ==== 4179+0+0 (crc 0 0 0) 0x7ffab40485f0 con 0x22dbc90 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.641+0000 7ffac1ffb640 1 -- 172.21.15.97:0/1343649856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=709}) -- 0x7ffab003da40 con 0x22dbc90 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.641+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> v2:172.21.15.190:6809/239686547 conn(0x22dd880 0x2362600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.641+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 --> v2:172.21.15.190:6809/239686547 -- osd_op(unknown.0.0:1 200.2 200:47cad802:::threemore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e708) -- 0x2363e00 con 0x22dd880 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.642+0000 7ffac8f9a640 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffab0037030 0x7ffab0039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.642+0000 7ffac8f9a640 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffab0037030 0x7ffab0039540 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.643+0000 7ffac9f9c640 1 --2- 172.21.15.97:0/1343649856 >> v2:172.21.15.190:6809/239686547 conn(0x22dd880 0x2362600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:40:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.647+0000 7ffac9f9c640 1 --2- 172.21.15.97:0/1343649856 >> v2:172.21.15.190:6809/239686547 conn(0x22dd880 0x2362600 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.660+0000 7ffac9f9c640 1 -- 172.21.15.97:0/1343649856 <== osd.2 v2:172.21.15.190:6809/239686547 1 ==== osd_op_reply(1 threemore [writefull 0~11812] v708'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7ffabc00c430 con 0x22dd880 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 >> v2:172.21.15.190:6809/239686547 conn(0x22dd880 msgr2=0x2362600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> v2:172.21.15.190:6809/239686547 conn(0x22dd880 0x2362600 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffab0037030 msgr2=0x7ffab0039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffab0037030 0x7ffab0039540 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 msgr2=0x2361bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2361bb0 crc :-1 s=READY pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 shutdown_connections 2024-03-27T08:40:59.654 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> v2:172.21.15.190:6809/239686547 conn(0x22dd880 0x2362600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.655 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7ffab0037030 0x7ffab0039540 unknown :-1 s=CLOSED pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.655 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 --2- 172.21.15.97:0/1343649856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x22dbc90 0x2361bb0 unknown :-1 s=CLOSED pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.655 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 >> 172.21.15.97:0/1343649856 conn(0x21f9ea0 msgr2=0x2376040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:59.655 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 shutdown_connections 2024-03-27T08:40:59.655 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.661+0000 7ffad163cec0 1 -- 172.21.15.97:0/1343649856 wait complete. 2024-03-27T08:40:59.655 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_bytes 0 2024-03-27T08:40:59.747 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.747+0000 7fa29e2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:59.748 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.757+0000 7fa29e2ed640 1 Processor -- start 2024-03-27T08:40:59.748 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.757+0000 7fa29e2ed640 1 -- start start 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa29e2ed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2981319c0 0x7fa298131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa29e2ed640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa29806ecf0 con 0x7fa29809ba10 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa29e2ed640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa29806ae50 con 0x7fa2981319c0 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa297fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2981319c0 0x7fa298131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa297fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2981319c0 0x7fa298131dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40586/0 (socket says 172.21.15.97:40586) 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa2977fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa29809ba10 0x7fa29809be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54602/0 (socket says 172.21.15.97:54602) 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa297fff640 1 -- 172.21.15.97:0/1680756176 learned_addr learned my addr 172.21.15.97:0/1680756176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa297fff640 1 -- 172.21.15.97:0/1680756176 >> v1:172.21.15.97:6789/0 conn(0x7fa29809ba10 legacy=0x7fa29809be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.758+0000 7fa297fff640 1 -- 172.21.15.97:0/1680756176 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa298131570 con 0x7fa2981319c0 2024-03-27T08:40:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.759+0000 7fa297fff640 1 --2- 172.21.15.97:0/1680756176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2981319c0 0x7fa298131dd0 crc :-1 s=READY pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b48cf1d453ae9e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.759+0000 7fa296ffd640 1 -- 172.21.15.97:0/1680756176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa288009e50 con 0x7fa2981319c0 2024-03-27T08:40:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.759+0000 7fa296ffd640 1 -- 172.21.15.97:0/1680756176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa28800e230 con 0x7fa2981319c0 2024-03-27T08:40:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.759+0000 7fa296ffd640 1 -- 172.21.15.97:0/1680756176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa28800e4d0 con 0x7fa2981319c0 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.759+0000 7fa29e2ed640 1 -- 172.21.15.97:0/1680756176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2981319c0 msgr2=0x7fa298131dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.759+0000 7fa29e2ed640 1 --2- 172.21.15.97:0/1680756176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2981319c0 0x7fa298131dd0 crc :-1 s=READY pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.760+0000 7fa29e2ed640 1 -- 172.21.15.97:0/1680756176 shutdown_connections 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.760+0000 7fa29e2ed640 1 --2- 172.21.15.97:0/1680756176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2981319c0 0x7fa298131dd0 unknown :-1 s=CLOSED pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.760+0000 7fa29e2ed640 1 -- 172.21.15.97:0/1680756176 >> 172.21.15.97:0/1680756176 conn(0x7fa2980a1880 msgr2=0x7fa2980a1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.760+0000 7fa29e2ed640 1 -- 172.21.15.97:0/1680756176 shutdown_connections 2024-03-27T08:40:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.760+0000 7fa29e2ed640 1 -- 172.21.15.97:0/1680756176 wait complete. 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.760+0000 7fa29e2ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.761+0000 7fa29e2ed640 1 Processor -- start 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.761+0000 7fa29e2ed640 1 -- start start 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.762+0000 7fa29e2ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 0x7fa29811ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.762+0000 7fa29e2ed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa29812ed30 con 0x7fa29809ba10 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.762+0000 7fa297fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 0x7fa29811ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.762+0000 7fa297fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 0x7fa29811ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40594/0 (socket says 172.21.15.97:40594) 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.762+0000 7fa297fff640 1 -- 172.21.15.97:0/2977617785 learned_addr learned my addr 172.21.15.97:0/2977617785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa297fff640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29811ed90 con 0x7fa29809ba10 2024-03-27T08:40:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa297fff640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 0x7fa29811ad40 crc :-1 s=READY pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa28800eda0 con 0x7fa29809ba10 2024-03-27T08:40:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29811c8a0 con 0x7fa29809ba10 2024-03-27T08:40:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa288015420 con 0x7fa29809ba10 2024-03-27T08:40:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa2880156c0 con 0x7fa29809ba10 2024-03-27T08:40:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.763+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29811d800 con 0x7fa29809ba10 2024-03-27T08:40:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.764+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fa2880158c0 con 0x7fa29809ba10 2024-03-27T08:40:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.764+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2980693c0 con 0x7fa29809ba10 2024-03-27T08:40:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.764+0000 7fa294ff9640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa284038750 0x7fa28403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:40:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.764+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(708..708 src has 399..708) ==== 4179+0+0 (crc 0 0 0) 0x7fa288049c10 con 0x7fa29809ba10 2024-03-27T08:40:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.764+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=709}) -- 0x7fa28403f160 con 0x7fa29809ba10 2024-03-27T08:40:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.764+0000 7fa2977fe640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa284038750 0x7fa28403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:40:59.759 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.765+0000 7fa2977fe640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa284038750 0x7fa28403ac60 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:40:59.761 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.769+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fa288010be0 con 0x7fa29809ba10 2024-03-27T08:40:59.939 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:40:59.947+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"} v 0) -- 0x7fa298072990 con 0x7fa29809ba10 2024-03-27T08:41:00.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:00.864+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v709) ==== 217+0+0 (crc 0 0 0) 0x7fa2880107d0 con 0x7fa29809ba10 2024-03-27T08:41:00.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:00.865+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(709..709 src has 399..709) ==== 628+0+0 (crc 0 0 0) 0x7fa288010e10 con 0x7fa29809ba10 2024-03-27T08:41:00.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:00.865+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=710}) -- 0x7fa28403fee0 con 0x7fa29809ba10 2024-03-27T08:41:00.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:00.978+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"} v 0) -- 0x7fa29811d180 con 0x7fa29809ba10 2024-03-27T08:41:01.860 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.869+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v710) ==== 217+0+0 (crc 0 0 0) 0x7fa2880109b0 con 0x7fa29809ba10 2024-03-27T08:41:01.861 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_bytes = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:41:01.861 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.870+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(710..710 src has 399..710) ==== 628+0+0 (crc 0 0 0) 0x7fa288007d50 con 0x7fa29809ba10 2024-03-27T08:41:01.861 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.870+0000 7fa294ff9640 1 -- 172.21.15.97:0/2977617785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=711}) -- 0x7fa2840401c0 con 0x7fa29809ba10 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa284038750 msgr2=0x7fa28403ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa284038750 0x7fa28403ac60 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 msgr2=0x7fa29811ad40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 0x7fa29811ad40 crc :-1 s=READY pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 shutdown_connections 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa284038750 0x7fa28403ac60 unknown :-1 s=CLOSED pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:01.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 --2- 172.21.15.97:0/2977617785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29809ba10 0x7fa29811ad40 unknown :-1 s=CLOSED pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:01.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 >> 172.21.15.97:0/2977617785 conn(0x7fa2980a1880 msgr2=0x7fa29812fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:01.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 shutdown_connections 2024-03-27T08:41:01.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.871+0000 7fa29e2ed640 1 -- 172.21.15.97:0/2977617785 wait complete. 2024-03-27T08:41:01.873 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota 97a8ed02-de19-41fe-985b-30a778c6d918 max_objects 0 2024-03-27T08:41:01.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.965+0000 7fcdd79ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:01.965 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.974+0000 7fcdd679d640 1 Processor -- start 2024-03-27T08:41:01.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.975+0000 7fcdd679d640 1 -- start start 2024-03-27T08:41:01.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd679d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcdd00a1800 0x7fcdd00a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:01.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd679d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd00a2de0 con 0x7fcdd01345f0 2024-03-27T08:41:01.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd679d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd006dd60 con 0x7fcdd00a1800 2024-03-27T08:41:01.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd579b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcdd00a1800 0x7fcdd00a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:01.968 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd579b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcdd00a1800 0x7fcdd00a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40600/0 (socket says 172.21.15.97:40600) 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd579b640 1 -- 172.21.15.97:0/1806563889 learned_addr learned my addr 172.21.15.97:0/1806563889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd4f9a640 1 --1- 172.21.15.97:0/1806563889 >> v1:172.21.15.97:6789/0 conn(0x7fcdd01345f0 0x7fcdd0134a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54610/0 (socket says 172.21.15.97:54610) 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd579b640 1 -- 172.21.15.97:0/1806563889 >> v1:172.21.15.97:6789/0 conn(0x7fcdd01345f0 legacy=0x7fcdd0134a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd579b640 1 -- 172.21.15.97:0/1806563889 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd00a0e20 con 0x7fcdd00a1800 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdd579b640 1 --2- 172.21.15.97:0/1806563889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd00a1c10 crc :-1 s=READY pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b488dcc7388cae8b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdc7fff640 1 -- 172.21.15.97:0/1806563889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdcc009e50 con 0x7fcdd00a1800 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdc7fff640 1 -- 172.21.15.97:0/1806563889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcdcc00e230 con 0x7fcdd00a1800 2024-03-27T08:41:01.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.976+0000 7fcdc7fff640 1 -- 172.21.15.97:0/1806563889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdcc00e4d0 con 0x7fcdd00a1800 2024-03-27T08:41:01.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 -- 172.21.15.97:0/1806563889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 msgr2=0x7fcdd00a1c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:01.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 --2- 172.21.15.97:0/1806563889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd00a1c10 crc :-1 s=READY pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:01.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 -- 172.21.15.97:0/1806563889 shutdown_connections 2024-03-27T08:41:01.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 --2- 172.21.15.97:0/1806563889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd00a1c10 unknown :-1 s=CLOSED pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:01.970 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 -- 172.21.15.97:0/1806563889 >> 172.21.15.97:0/1806563889 conn(0x7fcdd00a4940 msgr2=0x7fcdd00a4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:01.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 -- 172.21.15.97:0/1806563889 shutdown_connections 2024-03-27T08:41:01.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 1 -- 172.21.15.97:0/1806563889 wait complete. 2024-03-27T08:41:01.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.977+0000 7fcdd679d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:01.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.979+0000 7fcdd679d640 1 Processor -- start 2024-03-27T08:41:01.971 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.979+0000 7fcdd679d640 1 -- start start 2024-03-27T08:41:01.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd679d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd011dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:01.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd679d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd011ef50 con 0x7fcdd00a1800 2024-03-27T08:41:01.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd579b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd011dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:01.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd579b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd011dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40606/0 (socket says 172.21.15.97:40606) 2024-03-27T08:41:01.972 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd579b640 1 -- 172.21.15.97:0/4085687712 learned_addr learned my addr 172.21.15.97:0/4085687712 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd579b640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd011f830 con 0x7fcdd00a1800 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd579b640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd011dc50 crc :-1 s=READY pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdcc00eda0 con 0x7fcdd00a1800 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcdcc015420 con 0x7fcdd00a1800 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd0075190 con 0x7fcdd00a1800 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.980+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdcc0156c0 con 0x7fcdd00a1800 2024-03-27T08:41:01.973 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.981+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01211c0 con 0x7fcdd00a1800 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.981+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fcdcc015870 con 0x7fcdd00a1800 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.982+0000 7fcdc5ffb640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fcdb40387a0 0x7fcdb403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.982+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(710..710 src has 399..710) ==== 4179+0+0 (crc 0 0 0) 0x7fcdcc049ae0 con 0x7fcdd00a1800 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.982+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd00693c0 con 0x7fcdd00a1800 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.982+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=711}) -- 0x7fcdb403f1b0 con 0x7fcdd00a1800 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.982+0000 7fcdd4f9a640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fcdb40387a0 0x7fcdb403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:01.974 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.982+0000 7fcdd4f9a640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fcdb40387a0 0x7fcdb403acb0 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:01.978 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:01.987+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fcdcc012c50 con 0x7fcdd00a1800 2024-03-27T08:41:02.154 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:02.162+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"} v 0) -- 0x7fcdd00695f0 con 0x7fcdd00a1800 2024-03-27T08:41:02.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:02.872+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v711) ==== 221+0+0 (crc 0 0 0) 0x7fcdcc012840 con 0x7fcdd00a1800 2024-03-27T08:41:02.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:02.872+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(711..711 src has 399..711) ==== 628+0+0 (crc 0 0 0) 0x7fcdcc012e80 con 0x7fcdd00a1800 2024-03-27T08:41:02.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:02.872+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=712}) -- 0x7fcdb403fec0 con 0x7fcdd00a1800 2024-03-27T08:41:02.969 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:02.978+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"} v 0) -- 0x7fcdd0120790 con 0x7fcdd00a1800 2024-03-27T08:41:03.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.873+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 v712) ==== 221+0+0 (crc 0 0 0) 0x7fcdcc012a20 con 0x7fcdd00a1800 2024-03-27T08:41:03.865 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 0 for pool 97a8ed02-de19-41fe-985b-30a778c6d918 2024-03-27T08:41:03.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.873+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(712..712 src has 399..712) ==== 628+0+0 (crc 0 0 0) 0x7fcdcc013e10 con 0x7fcdd00a1800 2024-03-27T08:41:03.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.873+0000 7fcdc5ffb640 1 -- 172.21.15.97:0/4085687712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=713}) -- 0x7fcdb40401a0 con 0x7fcdd00a1800 2024-03-27T08:41:03.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.875+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fcdb40387a0 msgr2=0x7fcdb403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:03.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.875+0000 7fcdd679d640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fcdb40387a0 0x7fcdb403acb0 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.875+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 msgr2=0x7fcdd011dc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.875+0000 7fcdd679d640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd011dc50 crc :-1 s=READY pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.876+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 shutdown_connections 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.876+0000 7fcdd679d640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fcdb40387a0 0x7fcdb403acb0 unknown :-1 s=CLOSED pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.876+0000 7fcdd679d640 1 --2- 172.21.15.97:0/4085687712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd00a1800 0x7fcdd011dc50 unknown :-1 s=CLOSED pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.876+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 >> 172.21.15.97:0/4085687712 conn(0x7fcdd00a4940 msgr2=0x7fcdd01bc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.876+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 shutdown_connections 2024-03-27T08:41:03.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:03.876+0000 7fcdd679d640 1 -- 172.21.15.97:0/4085687712 wait complete. 2024-03-27T08:41:03.878 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-27T08:41:33.880 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool delete 97a8ed02-de19-41fe-985b-30a778c6d918 97a8ed02-de19-41fe-985b-30a778c6d918 --yes-i-really-really-mean-it 2024-03-27T08:41:33.982 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.981+0000 7f207ecad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:33.983 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.991+0000 7f207ecad640 1 Processor -- start 2024-03-27T08:41:33.983 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.991+0000 7f207ecad640 1 -- start start 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f207ecad640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20780a0450 0x7f20780a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f207ecad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207806f8d0 con 0x7f20780a0da0 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f207ecad640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2078070d00 con 0x7f20780a0450 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f207ca49640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20780a0450 0x7f20780a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f2077fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f20780a0da0 0x7f207812e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57430/0 (socket says 172.21.15.97:57430) 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f2077fff640 1 -- 172.21.15.97:0/3422866418 learned_addr learned my addr 172.21.15.97:0/3422866418 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f207ca49640 1 -- 172.21.15.97:0/3422866418 >> v1:172.21.15.97:6789/0 conn(0x7f20780a0da0 legacy=0x7f207812e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f207ca49640 1 -- 172.21.15.97:0/3422866418 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20780a11b0 con 0x7f20780a0450 2024-03-27T08:41:33.985 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.992+0000 7f20777fe640 1 -- 172.21.15.97:0/3422866418 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193055845 0 0) 0x7f207806f8d0 con 0x7f20780a0da0 2024-03-27T08:41:33.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f207ca49640 1 --2- 172.21.15.97:0/3422866418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0450 0x7f20780a0860 crc :-1 s=READY pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab56ccc2f209343 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:33.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f20777fe640 1 -- 172.21.15.97:0/3422866418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f206c009d90 con 0x7f20780a0450 2024-03-27T08:41:33.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f20777fe640 1 -- 172.21.15.97:0/3422866418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f206c00e230 con 0x7f20780a0450 2024-03-27T08:41:33.986 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f20777fe640 1 -- 172.21.15.97:0/3422866418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f206c00e4d0 con 0x7f20780a0450 2024-03-27T08:41:33.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f207ecad640 1 -- 172.21.15.97:0/3422866418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0450 msgr2=0x7f20780a0860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:33.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.993+0000 7f207ecad640 1 --2- 172.21.15.97:0/3422866418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0450 0x7f20780a0860 crc :-1 s=READY pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:33.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.994+0000 7f207ecad640 1 -- 172.21.15.97:0/3422866418 shutdown_connections 2024-03-27T08:41:33.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.994+0000 7f207ecad640 1 --2- 172.21.15.97:0/3422866418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0450 0x7f20780a0860 unknown :-1 s=CLOSED pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:33.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.994+0000 7f207ecad640 1 -- 172.21.15.97:0/3422866418 >> 172.21.15.97:0/3422866418 conn(0x7f20780a5710 msgr2=0x7f20780a1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:33.987 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.994+0000 7f207ecad640 1 -- 172.21.15.97:0/3422866418 shutdown_connections 2024-03-27T08:41:33.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.994+0000 7f207ecad640 1 -- 172.21.15.97:0/3422866418 wait complete. 2024-03-27T08:41:33.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.994+0000 7f207ecad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:33.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.995+0000 7f207ecad640 1 Processor -- start 2024-03-27T08:41:33.988 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.995+0000 7f207ecad640 1 -- start start 2024-03-27T08:41:33.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.996+0000 7f207ecad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 0x7f207811f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:33.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.996+0000 7f207ecad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f207811ee30 con 0x7f20780a0da0 2024-03-27T08:41:33.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ca49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 0x7f207811f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:33.989 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ca49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 0x7f207811f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52468/0 (socket says 172.21.15.97:52468) 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ca49640 1 -- 172.21.15.97:0/198080968 learned_addr learned my addr 172.21.15.97:0/198080968 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ca49640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207812ffb0 con 0x7f20780a0da0 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ca49640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 0x7f207811f360 crc :-1 s=READY pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f206c013610 con 0x7f20780a0da0 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f206c013770 con 0x7f20780a0da0 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207811ea90 con 0x7f20780a0da0 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f206c013a10 con 0x7f20780a0da0 2024-03-27T08:41:33.990 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.997+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781224a0 con 0x7f20780a0da0 2024-03-27T08:41:33.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.998+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f206c013bc0 con 0x7f20780a0da0 2024-03-27T08:41:33.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.998+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207811ec80 con 0x7f20780a0da0 2024-03-27T08:41:33.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.998+0000 7f20757fa640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f205c0387a0 0x7f205c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:33.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.998+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(712..712 src has 399..712) ==== 4179+0+0 (crc 0 0 0) 0x7f206c018040 con 0x7f20780a0da0 2024-03-27T08:41:33.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.998+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=713}) -- 0x7f205c03f150 con 0x7f20780a0da0 2024-03-27T08:41:33.992 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.999+0000 7f2077fff640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f205c0387a0 0x7f205c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:33.992 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:33.999+0000 7f2077fff640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f205c0387a0 0x7f205c03acb0 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:33.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.004+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f206c011380 con 0x7f20780a0da0 2024-03-27T08:41:34.173 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.181+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pool2": "97a8ed02-de19-41fe-985b-30a778c6d918", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20780693c0 con 0x7f20780a0da0 2024-03-27T08:41:34.857 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.865+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pool2": "97a8ed02-de19-41fe-985b-30a778c6d918", "yes_i_really_really_mean_it": true}]=0 pool '97a8ed02-de19-41fe-985b-30a778c6d918' removed v713) ==== 248+0+0 (crc 0 0 0) 0x7f206c00f2a0 con 0x7f20780a0da0 2024-03-27T08:41:34.857 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.865+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(713..713 src has 399..713) ==== 296+0+0 (crc 0 0 0) 0x7f206c010e30 con 0x7f20780a0da0 2024-03-27T08:41:34.857 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.865+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=714}) -- 0x7f205c03fc80 con 0x7f20780a0da0 2024-03-27T08:41:34.963 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.971+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pool2": "97a8ed02-de19-41fe-985b-30a778c6d918", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20780008d0 con 0x7f20780a0da0 2024-03-27T08:41:34.963 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.971+0000 7f20757fa640 1 -- 172.21.15.97:0/198080968 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "97a8ed02-de19-41fe-985b-30a778c6d918", "pool2": "97a8ed02-de19-41fe-985b-30a778c6d918", "yes_i_really_really_mean_it": true}]=0 pool '97a8ed02-de19-41fe-985b-30a778c6d918' does not exist v713) ==== 255+0+0 (crc 0 0 0) 0x7f206c018a50 con 0x7f20780a0da0 2024-03-27T08:41:34.963 INFO:tasks.workunit.client.0.smithi097.stderr:pool '97a8ed02-de19-41fe-985b-30a778c6d918' does not exist 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.973+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f205c0387a0 msgr2=0x7f205c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.973+0000 7f207ecad640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f205c0387a0 0x7f205c03acb0 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.973+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 msgr2=0x7f207811f360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.973+0000 7f207ecad640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 0x7f207811f360 crc :-1 s=READY pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.974+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 shutdown_connections 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.974+0000 7f207ecad640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f205c0387a0 0x7f205c03acb0 unknown :-1 s=CLOSED pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.974+0000 7f207ecad640 1 --2- 172.21.15.97:0/198080968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20780a0da0 0x7f207811f360 unknown :-1 s=CLOSED pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.974+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 >> 172.21.15.97:0/198080968 conn(0x7f20780a0450 msgr2=0x7f20780a33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:34.966 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.974+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 shutdown_connections 2024-03-27T08:41:34.967 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:34.974+0000 7f207ecad640 1 -- 172.21.15.97:0/198080968 wait complete. 2024-03-27T08:41:34.975 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool delete 7165a426-c096-42b4-86ea-e81ff2d1e62a 7165a426-c096-42b4-86ea-e81ff2d1e62a --yes-i-really-really-mean-it 2024-03-27T08:41:35.067 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.067+0000 7fda6182d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:35.068 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.076+0000 7fda6182d640 1 Processor -- start 2024-03-27T08:41:35.068 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.076+0000 7fda6182d640 1 -- start start 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.077+0000 7fda6182d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda5c0710c0 0x7fda5c137de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.077+0000 7fda6182d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5c0719b0 con 0x7fda5c0705a0 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.077+0000 7fda6182d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fda5c071ef0 con 0x7fda5c0710c0 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda53fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda5c0710c0 0x7fda5c137de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda5affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda5c0705a0 0x7fda5c0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57438/0 (socket says 172.21.15.97:57438) 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda5affd640 1 -- 172.21.15.97:0/1649955202 learned_addr learned my addr 172.21.15.97:0/1649955202 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:35.070 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda53fff640 1 -- 172.21.15.97:0/1649955202 >> v1:172.21.15.97:6789/0 conn(0x7fda5c0705a0 legacy=0x7fda5c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:35.071 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda53fff640 1 -- 172.21.15.97:0/1649955202 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c071690 con 0x7fda5c0710c0 2024-03-27T08:41:35.071 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda53fff640 1 --2- 172.21.15.97:0/1649955202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c137de0 crc :-1 s=READY pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b7b53948d9343d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:35.071 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda5a7fc640 1 -- 172.21.15.97:0/1649955202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fda44009d90 con 0x7fda5c0710c0 2024-03-27T08:41:35.071 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda5a7fc640 1 -- 172.21.15.97:0/1649955202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fda4400e230 con 0x7fda5c0710c0 2024-03-27T08:41:35.071 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.078+0000 7fda5a7fc640 1 -- 172.21.15.97:0/1649955202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fda4400e4d0 con 0x7fda5c0710c0 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 -- 172.21.15.97:0/1649955202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 msgr2=0x7fda5c137de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 --2- 172.21.15.97:0/1649955202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c137de0 crc :-1 s=READY pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 -- 172.21.15.97:0/1649955202 shutdown_connections 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 --2- 172.21.15.97:0/1649955202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c137de0 unknown :-1 s=CLOSED pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 -- 172.21.15.97:0/1649955202 >> 172.21.15.97:0/1649955202 conn(0x7fda5c0a5710 msgr2=0x7fda5c0a1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 -- 172.21.15.97:0/1649955202 shutdown_connections 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 1 -- 172.21.15.97:0/1649955202 wait complete. 2024-03-27T08:41:35.072 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.079+0000 7fda6182d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:35.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.080+0000 7fda6182d640 1 Processor -- start 2024-03-27T08:41:35.073 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.081+0000 7fda6182d640 1 -- start start 2024-03-27T08:41:35.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda6182d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c11f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:35.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda6182d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c121750 con 0x7fda5c0710c0 2024-03-27T08:41:35.074 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda5affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c11f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda5affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c11f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52486/0 (socket says 172.21.15.97:52486) 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda5affd640 1 -- 172.21.15.97:0/936531839 learned_addr learned my addr 172.21.15.97:0/936531839 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda5affd640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c122af0 con 0x7fda5c0710c0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda5affd640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c11f590 crc :-1 s=READY pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fda4c005a60 con 0x7fda5c0710c0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fda4c005bc0 con 0x7fda5c0710c0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c11eb20 con 0x7fda5c0710c0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fda4c005e60 con 0x7fda5c0710c0 2024-03-27T08:41:35.075 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.082+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c120ee0 con 0x7fda5c0710c0 2024-03-27T08:41:35.076 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.083+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fda4c00c780 con 0x7fda5c0710c0 2024-03-27T08:41:35.076 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.083+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c0693c0 con 0x7fda5c0710c0 2024-03-27T08:41:35.077 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.083+0000 7fda58ff9640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fda3c0387a0 0x7fda3c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:35.077 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.083+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(713..713 src has 399..713) ==== 3791+0+0 (crc 0 0 0) 0x7fda4c048390 con 0x7fda5c0710c0 2024-03-27T08:41:35.077 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.084+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=714}) -- 0x7fda3c03ea10 con 0x7fda5c0710c0 2024-03-27T08:41:35.077 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.084+0000 7fda53fff640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fda3c0387a0 0x7fda3c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:35.077 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.084+0000 7fda53fff640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fda3c0387a0 0x7fda3c03acb0 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:35.080 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.089+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fda4c00d330 con 0x7fda5c0710c0 2024-03-27T08:41:35.270 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.278+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pool2": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda5c121dd0 con 0x7fda5c0710c0 2024-03-27T08:41:35.860 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.868+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pool2": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "yes_i_really_really_mean_it": true}]=0 pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' removed v714) ==== 248+0+0 (crc 0 0 0) 0x7fda4c00d150 con 0x7fda5c0710c0 2024-03-27T08:41:35.860 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.868+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(714..714 src has 399..714) ==== 296+0+0 (crc 0 0 0) 0x7fda4c00d510 con 0x7fda5c0710c0 2024-03-27T08:41:35.861 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.868+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=715}) -- 0x7fda3c03f6b0 con 0x7fda5c0710c0 2024-03-27T08:41:35.991 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:35.999+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pool2": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda5c0008d0 con 0x7fda5c0710c0 2024-03-27T08:41:35.992 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.000+0000 7fda58ff9640 1 -- 172.21.15.97:0/936531839 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "pool2": "7165a426-c096-42b4-86ea-e81ff2d1e62a", "yes_i_really_really_mean_it": true}]=0 pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' does not exist v714) ==== 255+0+0 (crc 0 0 0) 0x7fda4c0090b0 con 0x7fda5c0710c0 2024-03-27T08:41:35.992 INFO:tasks.workunit.client.0.smithi097.stderr:pool '7165a426-c096-42b4-86ea-e81ff2d1e62a' does not exist 2024-03-27T08:41:35.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fda3c0387a0 msgr2=0x7fda3c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:35.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fda3c0387a0 0x7fda3c03acb0 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:35.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 msgr2=0x7fda5c11f590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:35.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c11f590 crc :-1 s=READY pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:35.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 shutdown_connections 2024-03-27T08:41:35.995 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fda3c0387a0 0x7fda3c03acb0 unknown :-1 s=CLOSED pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:35.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 --2- 172.21.15.97:0/936531839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c0710c0 0x7fda5c11f590 unknown :-1 s=CLOSED pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:35.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 >> 172.21.15.97:0/936531839 conn(0x7fda5c0705a0 msgr2=0x7fda5c0a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:35.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 shutdown_connections 2024-03-27T08:41:35.996 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27T08:41:36.003+0000 7fda6182d640 1 -- 172.21.15.97:0/936531839 wait complete. 2024-03-27T08:41:36.011 INFO:tasks.workunit.client.0.smithi097.stdout:OK 2024-03-27T08:41:36.012 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo OK 2024-03-27T08:41:36.013 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-27T08:41:36.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-27T08:41:36.081 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-03-27T08:41:36.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-27T08:41:36.409 DEBUG:teuthology.parallel:result is None 2024-03-27T08:41:36.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T08:41:36.437 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T08:41:36.437 DEBUG:teuthology.orchestra.run.smithi097:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-27T08:41:36.499 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T08:41:36.499 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-27T08:41:36.512 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-27T08:41:36.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:41:36.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.653+0000 7f007d914640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:36.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.658+0000 7f007d914640 1 Processor -- start 2024-03-27T08:41:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.658+0000 7f007d914640 1 -- start start 2024-03-27T08:41:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f007d914640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0078134630 0x7f0078136a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f007d914640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f007806e090 con 0x7f007812e2c0 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f007d914640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f007809ef70 con 0x7f0078134630 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f0076ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f007812e2c0 0x7f007812e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57442/0 (socket says 172.21.15.97:57442) 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f0076ffd640 1 -- 172.21.15.97:0/1896894639 learned_addr learned my addr 172.21.15.97:0/1896894639 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f006e7fc640 1 --2- 172.21.15.97:0/1896894639 >> v2:172.21.15.97:3300/0 conn(0x7f0078134630 0x7f0078136a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f006e7fc640 1 -- 172.21.15.97:0/1896894639 >> v1:172.21.15.97:6789/0 conn(0x7f007812e2c0 legacy=0x7f007812e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f006e7fc640 1 -- 172.21.15.97:0/1896894639 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00780734e0 con 0x7f0078134630 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f006e7fc640 1 --2- 172.21.15.97:0/1896894639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0078134630 0x7f0078136a50 crc :-1 s=READY pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1f852d5644708 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.659+0000 7f00767fc640 1 -- 172.21.15.97:0/1896894639 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0060009d90 con 0x7f0078134630 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f00767fc640 1 -- 172.21.15.97:0/1896894639 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f006000e230 con 0x7f0078134630 2024-03-27T08:41:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f00767fc640 1 -- 172.21.15.97:0/1896894639 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f006000e4d0 con 0x7f0078134630 2024-03-27T08:41:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 -- 172.21.15.97:0/1896894639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0078134630 msgr2=0x7f0078136a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 --2- 172.21.15.97:0/1896894639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0078134630 0x7f0078136a50 crc :-1 s=READY pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 -- 172.21.15.97:0/1896894639 shutdown_connections 2024-03-27T08:41:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 --2- 172.21.15.97:0/1896894639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0078134630 0x7f0078136a50 unknown :-1 s=CLOSED pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 -- 172.21.15.97:0/1896894639 >> 172.21.15.97:0/1896894639 conn(0x7f007809d1d0 msgr2=0x7f007809d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 -- 172.21.15.97:0/1896894639 shutdown_connections 2024-03-27T08:41:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.660+0000 7f007d914640 1 -- 172.21.15.97:0/1896894639 wait complete. 2024-03-27T08:41:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.661+0000 7f007d914640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.662+0000 7f007d914640 1 Processor -- start 2024-03-27T08:41:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.662+0000 7f007d914640 1 -- start start 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.663+0000 7f007d914640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 0x7f0078119a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.663+0000 7f007d914640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f007806aa50 con 0x7f007812e2c0 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.663+0000 7f0076ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 0x7f0078119a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.663+0000 7f0076ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 0x7f0078119a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52496/0 (socket says 172.21.15.97:52496) 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.663+0000 7f0076ffd640 1 -- 172.21.15.97:0/629538553 learned_addr learned my addr 172.21.15.97:0/629538553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.663+0000 7f0076ffd640 1 -- 172.21.15.97:0/629538553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007811bf30 con 0x7f007812e2c0 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.664+0000 7f0076ffd640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 0x7f0078119a20 crc :-1 s=READY pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.664+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0068005a60 con 0x7f007812e2c0 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.664+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0068005bc0 con 0x7f007812e2c0 2024-03-27T08:41:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.664+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007811ac90 con 0x7f007812e2c0 2024-03-27T08:41:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.664+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0068005e60 con 0x7f007812e2c0 2024-03-27T08:41:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.664+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007811cf30 con 0x7f007812e2c0 2024-03-27T08:41:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.665+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0078063450 con 0x7f007812e2c0 2024-03-27T08:41:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.665+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f00680070b0 con 0x7f007812e2c0 2024-03-27T08:41:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.665+0000 7f0074ff9640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0058038750 0x7f005803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.665+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 399..714) ==== 3403+0+0 (crc 0 0 0) 0x7f0068048200 con 0x7f007812e2c0 2024-03-27T08:41:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.665+0000 7f006e7fc640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0058038750 0x7f005803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.666+0000 7f006e7fc640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0058038750 0x7f005803ac60 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.670+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f006800d510 con 0x7f007812e2c0 2024-03-27T08:41:36.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.843+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f007809ac90 con 0x7f0058038750 2024-03-27T08:41:36.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.848+0000 7f0074ff9640 1 -- 172.21.15.97:0/629538553 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7f0060009e80 con 0x7f0058038750 2024-03-27T08:41:36.840 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:41:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:41:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.849+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0058038750 msgr2=0x7f005803ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.849+0000 7f007d914640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0058038750 0x7f005803ac60 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.849+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 msgr2=0x7f0078119a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.849+0000 7f007d914640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 0x7f0078119a20 crc :-1 s=READY pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.850+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 shutdown_connections 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.850+0000 7f007d914640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f0058038750 0x7f005803ac60 unknown :-1 s=CLOSED pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.850+0000 7f007d914640 1 --2- 172.21.15.97:0/629538553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007812e2c0 0x7f0078119a20 unknown :-1 s=CLOSED pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.850+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 >> 172.21.15.97:0/629538553 conn(0x7f007809d1d0 msgr2=0x7f007812f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.850+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 shutdown_connections 2024-03-27T08:41:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.850+0000 7f007d914640 1 -- 172.21.15.97:0/629538553 wait complete. 2024-03-27T08:41:36.853 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":1263,"stamp":"2024-03-27T08:41:36.185260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":313236,"kb_used_data":15284,"kb_used_omap":0,"kb_used_meta":297920,"kb_avail":419117164,"statfs":{"total":429496729600,"available":429175975936,"internally_reserved":0,"allocated":15650816,"data_stored":7011571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":305070080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002403"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.592585+0000","last_change":"2024-03-27T08:27:58.956969+0000","last_active":"2024-03-27T08:41:05.592585+0000","last_peered":"2024-03-27T08:41:05.592585+0000","last_clean":"2024-03-27T08:41:05.592585+0000","last_became_active":"2024-03-27T08:27:58.944924+0000","last_became_peered":"2024-03-27T08:27:58.944924+0000","last_unstale":"2024-03-27T08:41:05.592585+0000","last_undegraded":"2024-03-27T08:41:05.592585+0000","last_fullsized":"2024-03-27T08:41:05.592585+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.937199+0000","last_change":"2024-03-27T08:27:59.042742+0000","last_active":"2024-03-27T08:41:03.937199+0000","last_peered":"2024-03-27T08:41:03.937199+0000","last_clean":"2024-03-27T08:41:03.937199+0000","last_became_active":"2024-03-27T08:27:59.029389+0000","last_became_peered":"2024-03-27T08:27:59.029389+0000","last_unstale":"2024-03-27T08:41:03.937199+0000","last_undegraded":"2024-03-27T08:41:03.937199+0000","last_fullsized":"2024-03-27T08:41:03.937199+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.617954+0000","last_change":"2024-03-27T08:27:58.946023+0000","last_active":"2024-03-27T08:41:05.617954+0000","last_peered":"2024-03-27T08:41:05.617954+0000","last_clean":"2024-03-27T08:41:05.617954+0000","last_became_active":"2024-03-27T08:27:58.934198+0000","last_became_peered":"2024-03-27T08:27:58.934198+0000","last_unstale":"2024-03-27T08:41:05.617954+0000","last_undegraded":"2024-03-27T08:41:05.617954+0000","last_fullsized":"2024-03-27T08:41:05.617954+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.699460+0000","last_change":"2024-03-27T08:27:58.982774+0000","last_active":"2024-03-27T08:41:05.699460+0000","last_peered":"2024-03-27T08:41:05.699460+0000","last_clean":"2024-03-27T08:41:05.699460+0000","last_became_active":"2024-03-27T08:27:58.963619+0000","last_became_peered":"2024-03-27T08:27:58.963619+0000","last_unstale":"2024-03-27T08:41:05.699460+0000","last_undegraded":"2024-03-27T08:41:05.699460+0000","last_fullsized":"2024-03-27T08:41:05.699460+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.586188+0000","last_change":"2024-03-27T08:27:58.974218+0000","last_active":"2024-03-27T08:41:05.586188+0000","last_peered":"2024-03-27T08:41:05.586188+0000","last_clean":"2024-03-27T08:41:05.586188+0000","last_became_active":"2024-03-27T08:27:58.967369+0000","last_became_peered":"2024-03-27T08:27:58.967369+0000","last_unstale":"2024-03-27T08:41:05.586188+0000","last_undegraded":"2024-03-27T08:41:05.586188+0000","last_fullsized":"2024-03-27T08:41:05.586188+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.945771+0000","last_change":"2024-03-27T08:27:58.970092+0000","last_active":"2024-03-27T08:41:03.945771+0000","last_peered":"2024-03-27T08:41:03.945771+0000","last_clean":"2024-03-27T08:41:03.945771+0000","last_became_active":"2024-03-27T08:27:58.956201+0000","last_became_peered":"2024-03-27T08:27:58.956201+0000","last_unstale":"2024-03-27T08:41:03.945771+0000","last_undegraded":"2024-03-27T08:41:03.945771+0000","last_fullsized":"2024-03-27T08:41:03.945771+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.925057+0000","last_change":"2024-03-27T08:27:59.003207+0000","last_active":"2024-03-27T08:41:03.925057+0000","last_peered":"2024-03-27T08:41:03.925057+0000","last_clean":"2024-03-27T08:41:03.925057+0000","last_became_active":"2024-03-27T08:27:58.981709+0000","last_became_peered":"2024-03-27T08:27:58.981709+0000","last_unstale":"2024-03-27T08:41:03.925057+0000","last_undegraded":"2024-03-27T08:41:03.925057+0000","last_fullsized":"2024-03-27T08:41:03.925057+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.903633+0000","last_change":"2024-03-27T08:27:58.997466+0000","last_active":"2024-03-27T08:41:03.903633+0000","last_peered":"2024-03-27T08:41:03.903633+0000","last_clean":"2024-03-27T08:41:03.903633+0000","last_became_active":"2024-03-27T08:27:58.977694+0000","last_became_peered":"2024-03-27T08:27:58.977694+0000","last_unstale":"2024-03-27T08:41:03.903633+0000","last_undegraded":"2024-03-27T08:41:03.903633+0000","last_fullsized":"2024-03-27T08:41:03.903633+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.644832+0000","last_change":"2024-03-27T08:27:59.001487+0000","last_active":"2024-03-27T08:41:05.644832+0000","last_peered":"2024-03-27T08:41:05.644832+0000","last_clean":"2024-03-27T08:41:05.644832+0000","last_became_active":"2024-03-27T08:27:58.979333+0000","last_became_peered":"2024-03-27T08:27:58.979333+0000","last_unstale":"2024-03-27T08:41:05.644832+0000","last_undegraded":"2024-03-27T08:41:05.644832+0000","last_fullsized":"2024-03-27T08:41:05.644832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.588534+0000","last_change":"2024-03-27T08:27:58.954275+0000","last_active":"2024-03-27T08:41:05.588534+0000","last_peered":"2024-03-27T08:41:05.588534+0000","last_clean":"2024-03-27T08:41:05.588534+0000","last_became_active":"2024-03-27T08:27:58.939180+0000","last_became_peered":"2024-03-27T08:27:58.939180+0000","last_unstale":"2024-03-27T08:41:05.588534+0000","last_undegraded":"2024-03-27T08:41:05.588534+0000","last_fullsized":"2024-03-27T08:41:05.588534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.658045+0000","last_change":"2024-03-27T08:27:58.988220+0000","last_active":"2024-03-27T08:41:05.658045+0000","last_peered":"2024-03-27T08:41:05.658045+0000","last_clean":"2024-03-27T08:41:05.658045+0000","last_became_active":"2024-03-27T08:27:58.955512+0000","last_became_peered":"2024-03-27T08:27:58.955512+0000","last_unstale":"2024-03-27T08:41:05.658045+0000","last_undegraded":"2024-03-27T08:41:05.658045+0000","last_fullsized":"2024-03-27T08:41:05.658045+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.916735+0000","last_change":"2024-03-27T08:27:59.103582+0000","last_active":"2024-03-27T08:41:03.916735+0000","last_peered":"2024-03-27T08:41:03.916735+0000","last_clean":"2024-03-27T08:41:03.916735+0000","last_became_active":"2024-03-27T08:27:59.098593+0000","last_became_peered":"2024-03-27T08:27:59.098593+0000","last_unstale":"2024-03-27T08:41:03.916735+0000","last_undegraded":"2024-03-27T08:41:03.916735+0000","last_fullsized":"2024-03-27T08:41:03.916735+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.693701+0000","last_change":"2024-03-27T08:27:58.999586+0000","last_active":"2024-03-27T08:41:05.693701+0000","last_peered":"2024-03-27T08:41:05.693701+0000","last_clean":"2024-03-27T08:41:05.693701+0000","last_became_active":"2024-03-27T08:27:58.973352+0000","last_became_peered":"2024-03-27T08:27:58.973352+0000","last_unstale":"2024-03-27T08:41:05.693701+0000","last_undegraded":"2024-03-27T08:41:05.693701+0000","last_fullsized":"2024-03-27T08:41:05.693701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.690909+0000","last_change":"2024-03-27T08:27:59.070788+0000","last_active":"2024-03-27T08:41:05.690909+0000","last_peered":"2024-03-27T08:41:05.690909+0000","last_clean":"2024-03-27T08:41:05.690909+0000","last_became_active":"2024-03-27T08:27:59.065991+0000","last_became_peered":"2024-03-27T08:27:59.065991+0000","last_unstale":"2024-03-27T08:41:05.690909+0000","last_undegraded":"2024-03-27T08:41:05.690909+0000","last_fullsized":"2024-03-27T08:41:05.690909+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.908630+0000","last_change":"2024-03-27T08:27:59.032231+0000","last_active":"2024-03-27T08:41:03.908630+0000","last_peered":"2024-03-27T08:41:03.908630+0000","last_clean":"2024-03-27T08:41:03.908630+0000","last_became_active":"2024-03-27T08:27:59.007454+0000","last_became_peered":"2024-03-27T08:27:59.007454+0000","last_unstale":"2024-03-27T08:41:03.908630+0000","last_undegraded":"2024-03-27T08:41:03.908630+0000","last_fullsized":"2024-03-27T08:41:03.908630+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.911667+0000","last_change":"2024-03-27T08:27:59.036310+0000","last_active":"2024-03-27T08:41:03.911667+0000","last_peered":"2024-03-27T08:41:03.911667+0000","last_clean":"2024-03-27T08:41:03.911667+0000","last_became_active":"2024-03-27T08:27:59.015211+0000","last_became_peered":"2024-03-27T08:27:59.015211+0000","last_unstale":"2024-03-27T08:41:03.911667+0000","last_undegraded":"2024-03-27T08:41:03.911667+0000","last_fullsized":"2024-03-27T08:41:03.911667+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.913897+0000","last_change":"2024-03-27T08:27:58.973584+0000","last_active":"2024-03-27T08:41:03.913897+0000","last_peered":"2024-03-27T08:41:03.913897+0000","last_clean":"2024-03-27T08:41:03.913897+0000","last_became_active":"2024-03-27T08:27:58.960673+0000","last_became_peered":"2024-03-27T08:27:58.960673+0000","last_unstale":"2024-03-27T08:41:03.913897+0000","last_undegraded":"2024-03-27T08:41:03.913897+0000","last_fullsized":"2024-03-27T08:41:03.913897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.701323+0000","last_change":"2024-03-27T08:27:59.040676+0000","last_active":"2024-03-27T08:41:05.701323+0000","last_peered":"2024-03-27T08:41:05.701323+0000","last_clean":"2024-03-27T08:41:05.701323+0000","last_became_active":"2024-03-27T08:27:59.034744+0000","last_became_peered":"2024-03-27T08:27:59.034744+0000","last_unstale":"2024-03-27T08:41:05.701323+0000","last_undegraded":"2024-03-27T08:41:05.701323+0000","last_fullsized":"2024-03-27T08:41:05.701323+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.623255+0000","last_change":"2024-03-27T08:27:58.993514+0000","last_active":"2024-03-27T08:41:05.623255+0000","last_peered":"2024-03-27T08:41:05.623255+0000","last_clean":"2024-03-27T08:41:05.623255+0000","last_became_active":"2024-03-27T08:27:58.988938+0000","last_became_peered":"2024-03-27T08:27:58.988938+0000","last_unstale":"2024-03-27T08:41:05.623255+0000","last_undegraded":"2024-03-27T08:41:05.623255+0000","last_fullsized":"2024-03-27T08:41:05.623255+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.637259+0000","last_change":"2024-03-27T08:27:58.953456+0000","last_active":"2024-03-27T08:41:05.637259+0000","last_peered":"2024-03-27T08:41:05.637259+0000","last_clean":"2024-03-27T08:41:05.637259+0000","last_became_active":"2024-03-27T08:27:58.940428+0000","last_became_peered":"2024-03-27T08:27:58.940428+0000","last_unstale":"2024-03-27T08:41:05.637259+0000","last_undegraded":"2024-03-27T08:41:05.637259+0000","last_fullsized":"2024-03-27T08:41:05.637259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.617111+0000","last_change":"2024-03-27T08:27:58.991121+0000","last_active":"2024-03-27T08:41:05.617111+0000","last_peered":"2024-03-27T08:41:05.617111+0000","last_clean":"2024-03-27T08:41:05.617111+0000","last_became_active":"2024-03-27T08:27:58.982971+0000","last_became_peered":"2024-03-27T08:27:58.982971+0000","last_unstale":"2024-03-27T08:41:05.617111+0000","last_undegraded":"2024-03-27T08:41:05.617111+0000","last_fullsized":"2024-03-27T08:41:05.617111+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.621935+0000","last_change":"2024-03-27T08:27:59.023638+0000","last_active":"2024-03-27T08:41:05.621935+0000","last_peered":"2024-03-27T08:41:05.621935+0000","last_clean":"2024-03-27T08:41:05.621935+0000","last_became_active":"2024-03-27T08:27:59.017861+0000","last_became_peered":"2024-03-27T08:27:59.017861+0000","last_unstale":"2024-03-27T08:41:05.621935+0000","last_undegraded":"2024-03-27T08:41:05.621935+0000","last_fullsized":"2024-03-27T08:41:05.621935+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.587067+0000","last_change":"2024-03-27T08:27:58.950095+0000","last_active":"2024-03-27T08:41:05.587067+0000","last_peered":"2024-03-27T08:41:05.587067+0000","last_clean":"2024-03-27T08:41:05.587067+0000","last_became_active":"2024-03-27T08:27:58.942454+0000","last_became_peered":"2024-03-27T08:27:58.942454+0000","last_unstale":"2024-03-27T08:41:05.587067+0000","last_undegraded":"2024-03-27T08:41:05.587067+0000","last_fullsized":"2024-03-27T08:41:05.587067+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.652353+0000","last_change":"2024-03-27T08:27:58.986152+0000","last_active":"2024-03-27T08:41:05.652353+0000","last_peered":"2024-03-27T08:41:05.652353+0000","last_clean":"2024-03-27T08:41:05.652353+0000","last_became_active":"2024-03-27T08:27:58.960185+0000","last_became_peered":"2024-03-27T08:27:58.960185+0000","last_unstale":"2024-03-27T08:41:05.652353+0000","last_undegraded":"2024-03-27T08:41:05.652353+0000","last_fullsized":"2024-03-27T08:41:05.652353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.698522+0000","last_change":"2024-03-27T08:27:58.996597+0000","last_active":"2024-03-27T08:41:05.698522+0000","last_peered":"2024-03-27T08:41:05.698522+0000","last_clean":"2024-03-27T08:41:05.698522+0000","last_became_active":"2024-03-27T08:27:58.969756+0000","last_became_peered":"2024-03-27T08:27:58.969756+0000","last_unstale":"2024-03-27T08:41:05.698522+0000","last_undegraded":"2024-03-27T08:41:05.698522+0000","last_fullsized":"2024-03-27T08:41:05.698522+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.936336+0000","last_change":"2024-03-27T08:27:59.018353+0000","last_active":"2024-03-27T08:41:03.936336+0000","last_peered":"2024-03-27T08:41:03.936336+0000","last_clean":"2024-03-27T08:41:03.936336+0000","last_became_active":"2024-03-27T08:27:59.000404+0000","last_became_peered":"2024-03-27T08:27:59.000404+0000","last_unstale":"2024-03-27T08:41:03.936336+0000","last_undegraded":"2024-03-27T08:41:03.936336+0000","last_fullsized":"2024-03-27T08:41:03.936336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.667330+0000","last_change":"2024-03-27T08:27:59.012490+0000","last_active":"2024-03-27T08:41:05.667330+0000","last_peered":"2024-03-27T08:41:05.667330+0000","last_clean":"2024-03-27T08:41:05.667330+0000","last_became_active":"2024-03-27T08:27:59.004002+0000","last_became_peered":"2024-03-27T08:27:59.004002+0000","last_unstale":"2024-03-27T08:41:05.667330+0000","last_undegraded":"2024-03-27T08:41:05.667330+0000","last_fullsized":"2024-03-27T08:41:05.667330+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.922123+0000","last_change":"2024-03-27T08:27:59.052618+0000","last_active":"2024-03-27T08:41:03.922123+0000","last_peered":"2024-03-27T08:41:03.922123+0000","last_clean":"2024-03-27T08:41:03.922123+0000","last_became_active":"2024-03-27T08:27:59.045303+0000","last_became_peered":"2024-03-27T08:27:59.045303+0000","last_unstale":"2024-03-27T08:41:03.922123+0000","last_undegraded":"2024-03-27T08:41:03.922123+0000","last_fullsized":"2024-03-27T08:41:03.922123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.950979+0000","last_change":"2024-03-27T08:27:58.988986+0000","last_active":"2024-03-27T08:41:03.950979+0000","last_peered":"2024-03-27T08:41:03.950979+0000","last_clean":"2024-03-27T08:41:03.950979+0000","last_became_active":"2024-03-27T08:27:58.964911+0000","last_became_peered":"2024-03-27T08:27:58.964911+0000","last_unstale":"2024-03-27T08:41:03.950979+0000","last_undegraded":"2024-03-27T08:41:03.950979+0000","last_fullsized":"2024-03-27T08:41:03.950979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.941336+0000","last_change":"2024-03-27T08:27:59.011613+0000","last_active":"2024-03-27T08:41:03.941336+0000","last_peered":"2024-03-27T08:41:03.941336+0000","last_clean":"2024-03-27T08:41:03.941336+0000","last_became_active":"2024-03-27T08:27:58.985998+0000","last_became_peered":"2024-03-27T08:27:58.985998+0000","last_unstale":"2024-03-27T08:41:03.941336+0000","last_undegraded":"2024-03-27T08:41:03.941336+0000","last_fullsized":"2024-03-27T08:41:03.941336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1283,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.613949+0000","last_change":"2024-03-27T08:27:58.951044+0000","last_active":"2024-03-27T08:41:05.613949+0000","last_peered":"2024-03-27T08:41:05.613949+0000","last_clean":"2024-03-27T08:41:05.613949+0000","last_became_active":"2024-03-27T08:27:58.934959+0000","last_became_peered":"2024-03-27T08:27:58.934959+0000","last_unstale":"2024-03-27T08:41:05.613949+0000","last_undegraded":"2024-03-27T08:41:05.613949+0000","last_fullsized":"2024-03-27T08:41:05.613949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.611729+0000","last_change":"2024-03-27T08:27:58.978772+0000","last_active":"2024-03-27T08:41:05.611729+0000","last_peered":"2024-03-27T08:41:05.611729+0000","last_clean":"2024-03-27T08:41:05.611729+0000","last_became_active":"2024-03-27T08:27:58.972299+0000","last_became_peered":"2024-03-27T08:27:58.972299+0000","last_unstale":"2024-03-27T08:41:05.611729+0000","last_undegraded":"2024-03-27T08:41:05.611729+0000","last_fullsized":"2024-03-27T08:41:05.611729+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.612790+0000","last_change":"2024-03-27T08:23:09.066891+0000","last_active":"2024-03-27T08:41:05.612790+0000","last_peered":"2024-03-27T08:41:05.612790+0000","last_clean":"2024-03-27T08:41:05.612790+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:41:05.612790+0000","last_undegraded":"2024-03-27T08:41:05.612790+0000","last_fullsized":"2024-03-27T08:41:05.612790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1845,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.733759+0000","last_change":"2024-03-27T08:23:09.059885+0000","last_active":"2024-03-27T08:41:05.733759+0000","last_peered":"2024-03-27T08:41:05.733759+0000","last_clean":"2024-03-27T08:41:05.733759+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:41:05.733759+0000","last_undegraded":"2024-03-27T08:41:05.733759+0000","last_fullsized":"2024-03-27T08:41:05.733759+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1851,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.774666+0000","last_change":"2024-03-27T08:23:09.052784+0000","last_active":"2024-03-27T08:41:05.774666+0000","last_peered":"2024-03-27T08:41:05.774666+0000","last_clean":"2024-03-27T08:41:05.774666+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:41:05.774666+0000","last_undegraded":"2024-03-27T08:41:05.774666+0000","last_fullsized":"2024-03-27T08:41:05.774666+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1845,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.619459+0000","last_change":"2024-03-27T08:23:09.053884+0000","last_active":"2024-03-27T08:41:05.619459+0000","last_peered":"2024-03-27T08:41:05.619459+0000","last_clean":"2024-03-27T08:41:05.619459+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:41:05.619459+0000","last_undegraded":"2024-03-27T08:41:05.619459+0000","last_fullsized":"2024-03-27T08:41:05.619459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1887,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.976937+0000","last_change":"2024-03-27T08:23:09.061623+0000","last_active":"2024-03-27T08:41:03.976937+0000","last_peered":"2024-03-27T08:41:03.976937+0000","last_clean":"2024-03-27T08:41:03.976937+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:41:03.976937+0000","last_undegraded":"2024-03-27T08:41:03.976937+0000","last_fullsized":"2024-03-27T08:41:03.976937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.622848+0000","last_change":"2024-03-27T08:23:09.055461+0000","last_active":"2024-03-27T08:41:05.622848+0000","last_peered":"2024-03-27T08:41:05.622848+0000","last_clean":"2024-03-27T08:41:05.622848+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:41:05.622848+0000","last_undegraded":"2024-03-27T08:41:05.622848+0000","last_fullsized":"2024-03-27T08:41:05.622848+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1887,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.996706+0000","last_change":"2024-03-27T08:23:09.074614+0000","last_active":"2024-03-27T08:41:03.996706+0000","last_peered":"2024-03-27T08:41:03.996706+0000","last_clean":"2024-03-27T08:41:03.996706+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:41:03.996706+0000","last_undegraded":"2024-03-27T08:41:03.996706+0000","last_fullsized":"2024-03-27T08:41:03.996706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.688246+0000","last_change":"2024-03-27T08:23:09.065607+0000","last_active":"2024-03-27T08:41:05.688246+0000","last_peered":"2024-03-27T08:41:05.688246+0000","last_clean":"2024-03-27T08:41:05.688246+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:41:05.688246+0000","last_undegraded":"2024-03-27T08:41:05.688246+0000","last_fullsized":"2024-03-27T08:41:05.688246+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1984,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.692000+0000","last_change":"2024-03-27T08:23:07.031764+0000","last_active":"2024-03-27T08:41:05.692000+0000","last_peered":"2024-03-27T08:41:05.692000+0000","last_clean":"2024-03-27T08:41:05.692000+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:41:05.692000+0000","last_undegraded":"2024-03-27T08:41:05.692000+0000","last_fullsized":"2024-03-27T08:41:05.692000+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79604,"kb_used_data":4076,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104777996,"statfs":{"total":107374182400,"available":107292667904,"internally_reserved":0,"allocated":4173824,"data_stored":2026406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83260,"kb_used_data":3572,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104774340,"statfs":{"total":107374182400,"available":107288924160,"internally_reserved":0,"allocated":3657728,"data_stored":1464751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66568,"kb_used_data":3520,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791032,"statfs":{"total":107374182400,"available":107306016768,"internally_reserved":0,"allocated":3604480,"data_stored":1458999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83804,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773796,"statfs":{"total":107374182400,"available":107288367104,"internally_reserved":0,"allocated":4214784,"data_stored":2061415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:41:36.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:41:36.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.968+0000 7f2e67314640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.972+0000 7f2e67314640 1 Processor -- start 2024-03-27T08:41:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.972+0000 7f2e67314640 1 -- start start 2024-03-27T08:41:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.973+0000 7f2e67314640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e6012da20 0x7f2e6009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e67314640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e6012e2b0 con 0x7f2e6009c500 2024-03-27T08:41:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e67314640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6006e4d0 con 0x7f2e6012da20 2024-03-27T08:41:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e650b0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e6012da20 0x7f2e6009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e648af640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2e6009c500 0x7f2e6009c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57456/0 (socket says 172.21.15.97:57456) 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e648af640 1 -- 172.21.15.97:0/1671092351 learned_addr learned my addr 172.21.15.97:0/1671092351 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e650b0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e6012da20 0x7f2e6009aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52512/0 (socket says 172.21.15.97:52512) 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e650b0640 1 -- 172.21.15.97:0/1671092351 >> v1:172.21.15.97:6789/0 conn(0x7f2e6009c500 legacy=0x7f2e6009c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e650b0640 1 -- 172.21.15.97:0/1671092351 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6012dfa0 con 0x7f2e6012da20 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e650b0640 1 --2- 172.21.15.97:0/1671092351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6012da20 0x7f2e6009aa30 crc :-1 s=READY pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfb5a2869cb7bac server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e5ffff640 1 -- 172.21.15.97:0/1671092351 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e5000e040 con 0x7f2e6012da20 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e5ffff640 1 -- 172.21.15.97:0/1671092351 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2e50007670 con 0x7f2e6012da20 2024-03-27T08:41:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.974+0000 7f2e5ffff640 1 -- 172.21.15.97:0/1671092351 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e50007910 con 0x7f2e6012da20 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 -- 172.21.15.97:0/1671092351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6012da20 msgr2=0x7f2e6009aa30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 --2- 172.21.15.97:0/1671092351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6012da20 0x7f2e6009aa30 crc :-1 s=READY pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 -- 172.21.15.97:0/1671092351 shutdown_connections 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 --2- 172.21.15.97:0/1671092351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6012da20 0x7f2e6009aa30 unknown :-1 s=CLOSED pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 -- 172.21.15.97:0/1671092351 >> 172.21.15.97:0/1671092351 conn(0x7f2e6009d1d0 msgr2=0x7f2e6009d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 -- 172.21.15.97:0/1671092351 shutdown_connections 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 1 -- 172.21.15.97:0/1671092351 wait complete. 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.975+0000 7f2e67314640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.977+0000 7f2e67314640 1 Processor -- start 2024-03-27T08:41:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.977+0000 7f2e67314640 1 -- start start 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e67314640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 0x7f2e6011a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e67314640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6011d470 con 0x7f2e6009c500 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e650b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 0x7f2e6011a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e650b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 0x7f2e6011a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52524/0 (socket says 172.21.15.97:52524) 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e650b0640 1 -- 172.21.15.97:0/1683197495 learned_addr learned my addr 172.21.15.97:0/1683197495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e650b0640 1 -- 172.21.15.97:0/1683197495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6012abe0 con 0x7f2e6009c500 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e650b0640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 0x7f2e6011a050 crc :-1 s=READY pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e5000e040 con 0x7f2e6009c500 2024-03-27T08:41:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2e50012a90 con 0x7f2e6009c500 2024-03-27T08:41:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e601198f0 con 0x7f2e6009c500 2024-03-27T08:41:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.978+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e50012d30 con 0x7f2e6009c500 2024-03-27T08:41:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.979+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6011bbf0 con 0x7f2e6009c500 2024-03-27T08:41:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.979+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2e500153f0 con 0x7f2e6009c500 2024-03-27T08:41:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.979+0000 7f2e5dffb640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2e4c0387a0 0x7f2e4c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:36.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.979+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 399..714) ==== 3403+0+0 (crc 0 0 0) 0x7f2e50049bc0 con 0x7f2e6009c500 2024-03-27T08:41:36.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.980+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6011d5b0 con 0x7f2e6009c500 2024-03-27T08:41:36.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.980+0000 7f2e648af640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2e4c0387a0 0x7f2e4c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:36.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.980+0000 7f2e648af640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2e4c0387a0 0x7f2e4c03acb0 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:36.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:36.985+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f2e5000fe80 con 0x7f2e6009c500 2024-03-27T08:41:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.156+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e6006ecc0 con 0x7f2e4c0387a0 2024-03-27T08:41:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.160+0000 7f2e5dffb640 1 -- 172.21.15.97:0/1683197495 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7f2e54001250 con 0x7f2e4c0387a0 2024-03-27T08:41:37.153 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:41:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:41:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2e4c0387a0 msgr2=0x7f2e4c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2e4c0387a0 0x7f2e4c03acb0 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 msgr2=0x7f2e6011a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 0x7f2e6011a050 crc :-1 s=READY pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 shutdown_connections 2024-03-27T08:41:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f2e4c0387a0 0x7f2e4c03acb0 unknown :-1 s=CLOSED pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 --2- 172.21.15.97:0/1683197495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6009c500 0x7f2e6011a050 unknown :-1 s=CLOSED pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 >> 172.21.15.97:0/1683197495 conn(0x7f2e6009d1d0 msgr2=0x7f2e601307b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 shutdown_connections 2024-03-27T08:41:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.163+0000 7f2e67314640 1 -- 172.21.15.97:0/1683197495 wait complete. 2024-03-27T08:41:37.166 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":1263,"stamp":"2024-03-27T08:41:36.185260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":313236,"kb_used_data":15284,"kb_used_omap":0,"kb_used_meta":297920,"kb_avail":419117164,"statfs":{"total":429496729600,"available":429175975936,"internally_reserved":0,"allocated":15650816,"data_stored":7011571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":305070080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002403"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.592585+0000","last_change":"2024-03-27T08:27:58.956969+0000","last_active":"2024-03-27T08:41:05.592585+0000","last_peered":"2024-03-27T08:41:05.592585+0000","last_clean":"2024-03-27T08:41:05.592585+0000","last_became_active":"2024-03-27T08:27:58.944924+0000","last_became_peered":"2024-03-27T08:27:58.944924+0000","last_unstale":"2024-03-27T08:41:05.592585+0000","last_undegraded":"2024-03-27T08:41:05.592585+0000","last_fullsized":"2024-03-27T08:41:05.592585+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.937199+0000","last_change":"2024-03-27T08:27:59.042742+0000","last_active":"2024-03-27T08:41:03.937199+0000","last_peered":"2024-03-27T08:41:03.937199+0000","last_clean":"2024-03-27T08:41:03.937199+0000","last_became_active":"2024-03-27T08:27:59.029389+0000","last_became_peered":"2024-03-27T08:27:59.029389+0000","last_unstale":"2024-03-27T08:41:03.937199+0000","last_undegraded":"2024-03-27T08:41:03.937199+0000","last_fullsized":"2024-03-27T08:41:03.937199+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.617954+0000","last_change":"2024-03-27T08:27:58.946023+0000","last_active":"2024-03-27T08:41:05.617954+0000","last_peered":"2024-03-27T08:41:05.617954+0000","last_clean":"2024-03-27T08:41:05.617954+0000","last_became_active":"2024-03-27T08:27:58.934198+0000","last_became_peered":"2024-03-27T08:27:58.934198+0000","last_unstale":"2024-03-27T08:41:05.617954+0000","last_undegraded":"2024-03-27T08:41:05.617954+0000","last_fullsized":"2024-03-27T08:41:05.617954+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.699460+0000","last_change":"2024-03-27T08:27:58.982774+0000","last_active":"2024-03-27T08:41:05.699460+0000","last_peered":"2024-03-27T08:41:05.699460+0000","last_clean":"2024-03-27T08:41:05.699460+0000","last_became_active":"2024-03-27T08:27:58.963619+0000","last_became_peered":"2024-03-27T08:27:58.963619+0000","last_unstale":"2024-03-27T08:41:05.699460+0000","last_undegraded":"2024-03-27T08:41:05.699460+0000","last_fullsized":"2024-03-27T08:41:05.699460+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.586188+0000","last_change":"2024-03-27T08:27:58.974218+0000","last_active":"2024-03-27T08:41:05.586188+0000","last_peered":"2024-03-27T08:41:05.586188+0000","last_clean":"2024-03-27T08:41:05.586188+0000","last_became_active":"2024-03-27T08:27:58.967369+0000","last_became_peered":"2024-03-27T08:27:58.967369+0000","last_unstale":"2024-03-27T08:41:05.586188+0000","last_undegraded":"2024-03-27T08:41:05.586188+0000","last_fullsized":"2024-03-27T08:41:05.586188+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.945771+0000","last_change":"2024-03-27T08:27:58.970092+0000","last_active":"2024-03-27T08:41:03.945771+0000","last_peered":"2024-03-27T08:41:03.945771+0000","last_clean":"2024-03-27T08:41:03.945771+0000","last_became_active":"2024-03-27T08:27:58.956201+0000","last_became_peered":"2024-03-27T08:27:58.956201+0000","last_unstale":"2024-03-27T08:41:03.945771+0000","last_undegraded":"2024-03-27T08:41:03.945771+0000","last_fullsized":"2024-03-27T08:41:03.945771+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.925057+0000","last_change":"2024-03-27T08:27:59.003207+0000","last_active":"2024-03-27T08:41:03.925057+0000","last_peered":"2024-03-27T08:41:03.925057+0000","last_clean":"2024-03-27T08:41:03.925057+0000","last_became_active":"2024-03-27T08:27:58.981709+0000","last_became_peered":"2024-03-27T08:27:58.981709+0000","last_unstale":"2024-03-27T08:41:03.925057+0000","last_undegraded":"2024-03-27T08:41:03.925057+0000","last_fullsized":"2024-03-27T08:41:03.925057+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.903633+0000","last_change":"2024-03-27T08:27:58.997466+0000","last_active":"2024-03-27T08:41:03.903633+0000","last_peered":"2024-03-27T08:41:03.903633+0000","last_clean":"2024-03-27T08:41:03.903633+0000","last_became_active":"2024-03-27T08:27:58.977694+0000","last_became_peered":"2024-03-27T08:27:58.977694+0000","last_unstale":"2024-03-27T08:41:03.903633+0000","last_undegraded":"2024-03-27T08:41:03.903633+0000","last_fullsized":"2024-03-27T08:41:03.903633+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.644832+0000","last_change":"2024-03-27T08:27:59.001487+0000","last_active":"2024-03-27T08:41:05.644832+0000","last_peered":"2024-03-27T08:41:05.644832+0000","last_clean":"2024-03-27T08:41:05.644832+0000","last_became_active":"2024-03-27T08:27:58.979333+0000","last_became_peered":"2024-03-27T08:27:58.979333+0000","last_unstale":"2024-03-27T08:41:05.644832+0000","last_undegraded":"2024-03-27T08:41:05.644832+0000","last_fullsized":"2024-03-27T08:41:05.644832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.588534+0000","last_change":"2024-03-27T08:27:58.954275+0000","last_active":"2024-03-27T08:41:05.588534+0000","last_peered":"2024-03-27T08:41:05.588534+0000","last_clean":"2024-03-27T08:41:05.588534+0000","last_became_active":"2024-03-27T08:27:58.939180+0000","last_became_peered":"2024-03-27T08:27:58.939180+0000","last_unstale":"2024-03-27T08:41:05.588534+0000","last_undegraded":"2024-03-27T08:41:05.588534+0000","last_fullsized":"2024-03-27T08:41:05.588534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.658045+0000","last_change":"2024-03-27T08:27:58.988220+0000","last_active":"2024-03-27T08:41:05.658045+0000","last_peered":"2024-03-27T08:41:05.658045+0000","last_clean":"2024-03-27T08:41:05.658045+0000","last_became_active":"2024-03-27T08:27:58.955512+0000","last_became_peered":"2024-03-27T08:27:58.955512+0000","last_unstale":"2024-03-27T08:41:05.658045+0000","last_undegraded":"2024-03-27T08:41:05.658045+0000","last_fullsized":"2024-03-27T08:41:05.658045+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.916735+0000","last_change":"2024-03-27T08:27:59.103582+0000","last_active":"2024-03-27T08:41:03.916735+0000","last_peered":"2024-03-27T08:41:03.916735+0000","last_clean":"2024-03-27T08:41:03.916735+0000","last_became_active":"2024-03-27T08:27:59.098593+0000","last_became_peered":"2024-03-27T08:27:59.098593+0000","last_unstale":"2024-03-27T08:41:03.916735+0000","last_undegraded":"2024-03-27T08:41:03.916735+0000","last_fullsized":"2024-03-27T08:41:03.916735+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.693701+0000","last_change":"2024-03-27T08:27:58.999586+0000","last_active":"2024-03-27T08:41:05.693701+0000","last_peered":"2024-03-27T08:41:05.693701+0000","last_clean":"2024-03-27T08:41:05.693701+0000","last_became_active":"2024-03-27T08:27:58.973352+0000","last_became_peered":"2024-03-27T08:27:58.973352+0000","last_unstale":"2024-03-27T08:41:05.693701+0000","last_undegraded":"2024-03-27T08:41:05.693701+0000","last_fullsized":"2024-03-27T08:41:05.693701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.690909+0000","last_change":"2024-03-27T08:27:59.070788+0000","last_active":"2024-03-27T08:41:05.690909+0000","last_peered":"2024-03-27T08:41:05.690909+0000","last_clean":"2024-03-27T08:41:05.690909+0000","last_became_active":"2024-03-27T08:27:59.065991+0000","last_became_peered":"2024-03-27T08:27:59.065991+0000","last_unstale":"2024-03-27T08:41:05.690909+0000","last_undegraded":"2024-03-27T08:41:05.690909+0000","last_fullsized":"2024-03-27T08:41:05.690909+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.908630+0000","last_change":"2024-03-27T08:27:59.032231+0000","last_active":"2024-03-27T08:41:03.908630+0000","last_peered":"2024-03-27T08:41:03.908630+0000","last_clean":"2024-03-27T08:41:03.908630+0000","last_became_active":"2024-03-27T08:27:59.007454+0000","last_became_peered":"2024-03-27T08:27:59.007454+0000","last_unstale":"2024-03-27T08:41:03.908630+0000","last_undegraded":"2024-03-27T08:41:03.908630+0000","last_fullsized":"2024-03-27T08:41:03.908630+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.911667+0000","last_change":"2024-03-27T08:27:59.036310+0000","last_active":"2024-03-27T08:41:03.911667+0000","last_peered":"2024-03-27T08:41:03.911667+0000","last_clean":"2024-03-27T08:41:03.911667+0000","last_became_active":"2024-03-27T08:27:59.015211+0000","last_became_peered":"2024-03-27T08:27:59.015211+0000","last_unstale":"2024-03-27T08:41:03.911667+0000","last_undegraded":"2024-03-27T08:41:03.911667+0000","last_fullsized":"2024-03-27T08:41:03.911667+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.913897+0000","last_change":"2024-03-27T08:27:58.973584+0000","last_active":"2024-03-27T08:41:03.913897+0000","last_peered":"2024-03-27T08:41:03.913897+0000","last_clean":"2024-03-27T08:41:03.913897+0000","last_became_active":"2024-03-27T08:27:58.960673+0000","last_became_peered":"2024-03-27T08:27:58.960673+0000","last_unstale":"2024-03-27T08:41:03.913897+0000","last_undegraded":"2024-03-27T08:41:03.913897+0000","last_fullsized":"2024-03-27T08:41:03.913897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.701323+0000","last_change":"2024-03-27T08:27:59.040676+0000","last_active":"2024-03-27T08:41:05.701323+0000","last_peered":"2024-03-27T08:41:05.701323+0000","last_clean":"2024-03-27T08:41:05.701323+0000","last_became_active":"2024-03-27T08:27:59.034744+0000","last_became_peered":"2024-03-27T08:27:59.034744+0000","last_unstale":"2024-03-27T08:41:05.701323+0000","last_undegraded":"2024-03-27T08:41:05.701323+0000","last_fullsized":"2024-03-27T08:41:05.701323+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.623255+0000","last_change":"2024-03-27T08:27:58.993514+0000","last_active":"2024-03-27T08:41:05.623255+0000","last_peered":"2024-03-27T08:41:05.623255+0000","last_clean":"2024-03-27T08:41:05.623255+0000","last_became_active":"2024-03-27T08:27:58.988938+0000","last_became_peered":"2024-03-27T08:27:58.988938+0000","last_unstale":"2024-03-27T08:41:05.623255+0000","last_undegraded":"2024-03-27T08:41:05.623255+0000","last_fullsized":"2024-03-27T08:41:05.623255+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.637259+0000","last_change":"2024-03-27T08:27:58.953456+0000","last_active":"2024-03-27T08:41:05.637259+0000","last_peered":"2024-03-27T08:41:05.637259+0000","last_clean":"2024-03-27T08:41:05.637259+0000","last_became_active":"2024-03-27T08:27:58.940428+0000","last_became_peered":"2024-03-27T08:27:58.940428+0000","last_unstale":"2024-03-27T08:41:05.637259+0000","last_undegraded":"2024-03-27T08:41:05.637259+0000","last_fullsized":"2024-03-27T08:41:05.637259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.617111+0000","last_change":"2024-03-27T08:27:58.991121+0000","last_active":"2024-03-27T08:41:05.617111+0000","last_peered":"2024-03-27T08:41:05.617111+0000","last_clean":"2024-03-27T08:41:05.617111+0000","last_became_active":"2024-03-27T08:27:58.982971+0000","last_became_peered":"2024-03-27T08:27:58.982971+0000","last_unstale":"2024-03-27T08:41:05.617111+0000","last_undegraded":"2024-03-27T08:41:05.617111+0000","last_fullsized":"2024-03-27T08:41:05.617111+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.621935+0000","last_change":"2024-03-27T08:27:59.023638+0000","last_active":"2024-03-27T08:41:05.621935+0000","last_peered":"2024-03-27T08:41:05.621935+0000","last_clean":"2024-03-27T08:41:05.621935+0000","last_became_active":"2024-03-27T08:27:59.017861+0000","last_became_peered":"2024-03-27T08:27:59.017861+0000","last_unstale":"2024-03-27T08:41:05.621935+0000","last_undegraded":"2024-03-27T08:41:05.621935+0000","last_fullsized":"2024-03-27T08:41:05.621935+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.587067+0000","last_change":"2024-03-27T08:27:58.950095+0000","last_active":"2024-03-27T08:41:05.587067+0000","last_peered":"2024-03-27T08:41:05.587067+0000","last_clean":"2024-03-27T08:41:05.587067+0000","last_became_active":"2024-03-27T08:27:58.942454+0000","last_became_peered":"2024-03-27T08:27:58.942454+0000","last_unstale":"2024-03-27T08:41:05.587067+0000","last_undegraded":"2024-03-27T08:41:05.587067+0000","last_fullsized":"2024-03-27T08:41:05.587067+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.652353+0000","last_change":"2024-03-27T08:27:58.986152+0000","last_active":"2024-03-27T08:41:05.652353+0000","last_peered":"2024-03-27T08:41:05.652353+0000","last_clean":"2024-03-27T08:41:05.652353+0000","last_became_active":"2024-03-27T08:27:58.960185+0000","last_became_peered":"2024-03-27T08:27:58.960185+0000","last_unstale":"2024-03-27T08:41:05.652353+0000","last_undegraded":"2024-03-27T08:41:05.652353+0000","last_fullsized":"2024-03-27T08:41:05.652353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.698522+0000","last_change":"2024-03-27T08:27:58.996597+0000","last_active":"2024-03-27T08:41:05.698522+0000","last_peered":"2024-03-27T08:41:05.698522+0000","last_clean":"2024-03-27T08:41:05.698522+0000","last_became_active":"2024-03-27T08:27:58.969756+0000","last_became_peered":"2024-03-27T08:27:58.969756+0000","last_unstale":"2024-03-27T08:41:05.698522+0000","last_undegraded":"2024-03-27T08:41:05.698522+0000","last_fullsized":"2024-03-27T08:41:05.698522+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.936336+0000","last_change":"2024-03-27T08:27:59.018353+0000","last_active":"2024-03-27T08:41:03.936336+0000","last_peered":"2024-03-27T08:41:03.936336+0000","last_clean":"2024-03-27T08:41:03.936336+0000","last_became_active":"2024-03-27T08:27:59.000404+0000","last_became_peered":"2024-03-27T08:27:59.000404+0000","last_unstale":"2024-03-27T08:41:03.936336+0000","last_undegraded":"2024-03-27T08:41:03.936336+0000","last_fullsized":"2024-03-27T08:41:03.936336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.667330+0000","last_change":"2024-03-27T08:27:59.012490+0000","last_active":"2024-03-27T08:41:05.667330+0000","last_peered":"2024-03-27T08:41:05.667330+0000","last_clean":"2024-03-27T08:41:05.667330+0000","last_became_active":"2024-03-27T08:27:59.004002+0000","last_became_peered":"2024-03-27T08:27:59.004002+0000","last_unstale":"2024-03-27T08:41:05.667330+0000","last_undegraded":"2024-03-27T08:41:05.667330+0000","last_fullsized":"2024-03-27T08:41:05.667330+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.922123+0000","last_change":"2024-03-27T08:27:59.052618+0000","last_active":"2024-03-27T08:41:03.922123+0000","last_peered":"2024-03-27T08:41:03.922123+0000","last_clean":"2024-03-27T08:41:03.922123+0000","last_became_active":"2024-03-27T08:27:59.045303+0000","last_became_peered":"2024-03-27T08:27:59.045303+0000","last_unstale":"2024-03-27T08:41:03.922123+0000","last_undegraded":"2024-03-27T08:41:03.922123+0000","last_fullsized":"2024-03-27T08:41:03.922123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.950979+0000","last_change":"2024-03-27T08:27:58.988986+0000","last_active":"2024-03-27T08:41:03.950979+0000","last_peered":"2024-03-27T08:41:03.950979+0000","last_clean":"2024-03-27T08:41:03.950979+0000","last_became_active":"2024-03-27T08:27:58.964911+0000","last_became_peered":"2024-03-27T08:27:58.964911+0000","last_unstale":"2024-03-27T08:41:03.950979+0000","last_undegraded":"2024-03-27T08:41:03.950979+0000","last_fullsized":"2024-03-27T08:41:03.950979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.941336+0000","last_change":"2024-03-27T08:27:59.011613+0000","last_active":"2024-03-27T08:41:03.941336+0000","last_peered":"2024-03-27T08:41:03.941336+0000","last_clean":"2024-03-27T08:41:03.941336+0000","last_became_active":"2024-03-27T08:27:58.985998+0000","last_became_peered":"2024-03-27T08:27:58.985998+0000","last_unstale":"2024-03-27T08:41:03.941336+0000","last_undegraded":"2024-03-27T08:41:03.941336+0000","last_fullsized":"2024-03-27T08:41:03.941336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1283,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.613949+0000","last_change":"2024-03-27T08:27:58.951044+0000","last_active":"2024-03-27T08:41:05.613949+0000","last_peered":"2024-03-27T08:41:05.613949+0000","last_clean":"2024-03-27T08:41:05.613949+0000","last_became_active":"2024-03-27T08:27:58.934959+0000","last_became_peered":"2024-03-27T08:27:58.934959+0000","last_unstale":"2024-03-27T08:41:05.613949+0000","last_undegraded":"2024-03-27T08:41:05.613949+0000","last_fullsized":"2024-03-27T08:41:05.613949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.611729+0000","last_change":"2024-03-27T08:27:58.978772+0000","last_active":"2024-03-27T08:41:05.611729+0000","last_peered":"2024-03-27T08:41:05.611729+0000","last_clean":"2024-03-27T08:41:05.611729+0000","last_became_active":"2024-03-27T08:27:58.972299+0000","last_became_peered":"2024-03-27T08:27:58.972299+0000","last_unstale":"2024-03-27T08:41:05.611729+0000","last_undegraded":"2024-03-27T08:41:05.611729+0000","last_fullsized":"2024-03-27T08:41:05.611729+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.612790+0000","last_change":"2024-03-27T08:23:09.066891+0000","last_active":"2024-03-27T08:41:05.612790+0000","last_peered":"2024-03-27T08:41:05.612790+0000","last_clean":"2024-03-27T08:41:05.612790+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:41:05.612790+0000","last_undegraded":"2024-03-27T08:41:05.612790+0000","last_fullsized":"2024-03-27T08:41:05.612790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1845,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.733759+0000","last_change":"2024-03-27T08:23:09.059885+0000","last_active":"2024-03-27T08:41:05.733759+0000","last_peered":"2024-03-27T08:41:05.733759+0000","last_clean":"2024-03-27T08:41:05.733759+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:41:05.733759+0000","last_undegraded":"2024-03-27T08:41:05.733759+0000","last_fullsized":"2024-03-27T08:41:05.733759+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1851,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.774666+0000","last_change":"2024-03-27T08:23:09.052784+0000","last_active":"2024-03-27T08:41:05.774666+0000","last_peered":"2024-03-27T08:41:05.774666+0000","last_clean":"2024-03-27T08:41:05.774666+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:41:05.774666+0000","last_undegraded":"2024-03-27T08:41:05.774666+0000","last_fullsized":"2024-03-27T08:41:05.774666+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1845,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.619459+0000","last_change":"2024-03-27T08:23:09.053884+0000","last_active":"2024-03-27T08:41:05.619459+0000","last_peered":"2024-03-27T08:41:05.619459+0000","last_clean":"2024-03-27T08:41:05.619459+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:41:05.619459+0000","last_undegraded":"2024-03-27T08:41:05.619459+0000","last_fullsized":"2024-03-27T08:41:05.619459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1887,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.976937+0000","last_change":"2024-03-27T08:23:09.061623+0000","last_active":"2024-03-27T08:41:03.976937+0000","last_peered":"2024-03-27T08:41:03.976937+0000","last_clean":"2024-03-27T08:41:03.976937+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:41:03.976937+0000","last_undegraded":"2024-03-27T08:41:03.976937+0000","last_fullsized":"2024-03-27T08:41:03.976937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.622848+0000","last_change":"2024-03-27T08:23:09.055461+0000","last_active":"2024-03-27T08:41:05.622848+0000","last_peered":"2024-03-27T08:41:05.622848+0000","last_clean":"2024-03-27T08:41:05.622848+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:41:05.622848+0000","last_undegraded":"2024-03-27T08:41:05.622848+0000","last_fullsized":"2024-03-27T08:41:05.622848+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1887,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.996706+0000","last_change":"2024-03-27T08:23:09.074614+0000","last_active":"2024-03-27T08:41:03.996706+0000","last_peered":"2024-03-27T08:41:03.996706+0000","last_clean":"2024-03-27T08:41:03.996706+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:41:03.996706+0000","last_undegraded":"2024-03-27T08:41:03.996706+0000","last_fullsized":"2024-03-27T08:41:03.996706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.688246+0000","last_change":"2024-03-27T08:23:09.065607+0000","last_active":"2024-03-27T08:41:05.688246+0000","last_peered":"2024-03-27T08:41:05.688246+0000","last_clean":"2024-03-27T08:41:05.688246+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:41:05.688246+0000","last_undegraded":"2024-03-27T08:41:05.688246+0000","last_fullsized":"2024-03-27T08:41:05.688246+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1984,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.692000+0000","last_change":"2024-03-27T08:23:07.031764+0000","last_active":"2024-03-27T08:41:05.692000+0000","last_peered":"2024-03-27T08:41:05.692000+0000","last_clean":"2024-03-27T08:41:05.692000+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:41:05.692000+0000","last_undegraded":"2024-03-27T08:41:05.692000+0000","last_fullsized":"2024-03-27T08:41:05.692000+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79604,"kb_used_data":4076,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104777996,"statfs":{"total":107374182400,"available":107292667904,"internally_reserved":0,"allocated":4173824,"data_stored":2026406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83260,"kb_used_data":3572,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104774340,"statfs":{"total":107374182400,"available":107288924160,"internally_reserved":0,"allocated":3657728,"data_stored":1464751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66568,"kb_used_data":3520,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791032,"statfs":{"total":107374182400,"available":107306016768,"internally_reserved":0,"allocated":3604480,"data_stored":1458999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83804,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773796,"statfs":{"total":107374182400,"available":107288367104,"internally_reserved":0,"allocated":4214784,"data_stored":2061415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:41:37.168 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-27T08:41:37.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:41:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.281+0000 7f4fbc754640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.285+0000 7f4fbc754640 1 Processor -- start 2024-03-27T08:41:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.285+0000 7f4fbc754640 1 -- start start 2024-03-27T08:41:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.286+0000 7f4fbc754640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fb41378c0 0x7f4fb4139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.286+0000 7f4fbc754640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb40a06a0 con 0x7f4fb4131650 2024-03-27T08:41:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.286+0000 7f4fbc754640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb4132170 con 0x7f4fb41378c0 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb9cef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fb41378c0 0x7f4fb4139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb9cef640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fb41378c0 0x7f4fb4139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52528/0 (socket says 172.21.15.97:52528) 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fba4f0640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4fb4131650 0x7f4fb4131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57468/0 (socket says 172.21.15.97:57468) 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb9cef640 1 -- 172.21.15.97:0/2408556708 learned_addr learned my addr 172.21.15.97:0/2408556708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb9cef640 1 -- 172.21.15.97:0/2408556708 >> v1:172.21.15.97:6789/0 conn(0x7f4fb4131650 legacy=0x7f4fb4131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb9cef640 1 -- 172.21.15.97:0/2408556708 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb406de30 con 0x7f4fb41378c0 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb94ee640 1 -- 172.21.15.97:0/2408556708 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637701407 0 0) 0x7f4fb40a06a0 con 0x7f4fb4131650 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb9cef640 1 --2- 172.21.15.97:0/2408556708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb41378c0 0x7f4fb4139ce0 crc :-1 s=READY pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0c275effa9bd4d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb94ee640 1 -- 172.21.15.97:0/2408556708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4fa4009e50 con 0x7f4fb41378c0 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb94ee640 1 -- 172.21.15.97:0/2408556708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4fa400e230 con 0x7f4fb41378c0 2024-03-27T08:41:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.287+0000 7f4fb94ee640 1 -- 172.21.15.97:0/2408556708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4fa400e4d0 con 0x7f4fb41378c0 2024-03-27T08:41:37.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 -- 172.21.15.97:0/2408556708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb41378c0 msgr2=0x7f4fb4139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 --2- 172.21.15.97:0/2408556708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb41378c0 0x7f4fb4139ce0 crc :-1 s=READY pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 -- 172.21.15.97:0/2408556708 shutdown_connections 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 --2- 172.21.15.97:0/2408556708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb41378c0 0x7f4fb4139ce0 unknown :-1 s=CLOSED pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 -- 172.21.15.97:0/2408556708 >> 172.21.15.97:0/2408556708 conn(0x7f4fb40a15e0 msgr2=0x7f4fb40a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 -- 172.21.15.97:0/2408556708 shutdown_connections 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 1 -- 172.21.15.97:0/2408556708 wait complete. 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.288+0000 7f4fbc754640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.290+0000 7f4fbc754640 1 Processor -- start 2024-03-27T08:41:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.290+0000 7f4fbc754640 1 -- start start 2024-03-27T08:41:37.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fbc754640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 0x7f4fb411e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fbc754640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb411e140 con 0x7f4fb4131650 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fba4f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 0x7f4fb411e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fba4f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 0x7f4fb411e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52532/0 (socket says 172.21.15.97:52532) 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fba4f0640 1 -- 172.21.15.97:0/543349620 learned_addr learned my addr 172.21.15.97:0/543349620 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fba4f0640 1 -- 172.21.15.97:0/543349620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb411f710 con 0x7f4fb4131650 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fba4f0640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 0x7f4fb411e550 crc :-1 s=READY pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4fb0004cf0 con 0x7f4fb4131650 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4fb0004e50 con 0x7f4fb4131650 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb4121b90 con 0x7f4fb4131650 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4fb0007250 con 0x7f4fb4131650 2024-03-27T08:41:37.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.291+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb4120db0 con 0x7f4fb4131650 2024-03-27T08:41:37.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.292+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb411fff0 con 0x7f4fb4131650 2024-03-27T08:41:37.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.292+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f4fb0007450 con 0x7f4fb4131650 2024-03-27T08:41:37.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.293+0000 7f4faaffd640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4f98038750 0x7f4f9803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:37.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.293+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 399..714) ==== 3403+0+0 (crc 0 0 0) 0x7f4fb0048bc0 con 0x7f4fb4131650 2024-03-27T08:41:37.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.293+0000 7f4fb9cef640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4f98038750 0x7f4f9803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:37.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.293+0000 7f4fb9cef640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4f98038750 0x7f4f9803ac60 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:37.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.298+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f4fb00060b0 con 0x7f4fb4131650 2024-03-27T08:41:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.469+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fb40a0f20 con 0x7f4f98038750 2024-03-27T08:41:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.474+0000 7f4faaffd640 1 -- 172.21.15.97:0/543349620 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7f4fa4009e70 con 0x7f4f98038750 2024-03-27T08:41:37.465 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:41:37.470 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4f98038750 msgr2=0x7f4f9803ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4f98038750 0x7f4f9803ac60 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 msgr2=0x7f4fb411e550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 0x7f4fb411e550 crc :-1 s=READY pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 shutdown_connections 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4f98038750 0x7f4f9803ac60 unknown :-1 s=CLOSED pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 --2- 172.21.15.97:0/543349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb4131650 0x7f4fb411e550 unknown :-1 s=CLOSED pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 >> 172.21.15.97:0/543349620 conn(0x7f4fb40a15e0 msgr2=0x7f4fb41358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 shutdown_connections 2024-03-27T08:41:37.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.475+0000 7f4fbc754640 1 -- 172.21.15.97:0/543349620 wait complete. 2024-03-27T08:41:37.479 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":1263,"stamp":"2024-03-27T08:41:36.185260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":313236,"kb_used_data":15284,"kb_used_omap":0,"kb_used_meta":297920,"kb_avail":419117164,"statfs":{"total":429496729600,"available":429175975936,"internally_reserved":0,"allocated":15650816,"data_stored":7011571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":305070080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002403"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.592585+0000","last_change":"2024-03-27T08:27:58.956969+0000","last_active":"2024-03-27T08:41:05.592585+0000","last_peered":"2024-03-27T08:41:05.592585+0000","last_clean":"2024-03-27T08:41:05.592585+0000","last_became_active":"2024-03-27T08:27:58.944924+0000","last_became_peered":"2024-03-27T08:27:58.944924+0000","last_unstale":"2024-03-27T08:41:05.592585+0000","last_undegraded":"2024-03-27T08:41:05.592585+0000","last_fullsized":"2024-03-27T08:41:05.592585+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.937199+0000","last_change":"2024-03-27T08:27:59.042742+0000","last_active":"2024-03-27T08:41:03.937199+0000","last_peered":"2024-03-27T08:41:03.937199+0000","last_clean":"2024-03-27T08:41:03.937199+0000","last_became_active":"2024-03-27T08:27:59.029389+0000","last_became_peered":"2024-03-27T08:27:59.029389+0000","last_unstale":"2024-03-27T08:41:03.937199+0000","last_undegraded":"2024-03-27T08:41:03.937199+0000","last_fullsized":"2024-03-27T08:41:03.937199+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.617954+0000","last_change":"2024-03-27T08:27:58.946023+0000","last_active":"2024-03-27T08:41:05.617954+0000","last_peered":"2024-03-27T08:41:05.617954+0000","last_clean":"2024-03-27T08:41:05.617954+0000","last_became_active":"2024-03-27T08:27:58.934198+0000","last_became_peered":"2024-03-27T08:27:58.934198+0000","last_unstale":"2024-03-27T08:41:05.617954+0000","last_undegraded":"2024-03-27T08:41:05.617954+0000","last_fullsized":"2024-03-27T08:41:05.617954+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.699460+0000","last_change":"2024-03-27T08:27:58.982774+0000","last_active":"2024-03-27T08:41:05.699460+0000","last_peered":"2024-03-27T08:41:05.699460+0000","last_clean":"2024-03-27T08:41:05.699460+0000","last_became_active":"2024-03-27T08:27:58.963619+0000","last_became_peered":"2024-03-27T08:27:58.963619+0000","last_unstale":"2024-03-27T08:41:05.699460+0000","last_undegraded":"2024-03-27T08:41:05.699460+0000","last_fullsized":"2024-03-27T08:41:05.699460+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.586188+0000","last_change":"2024-03-27T08:27:58.974218+0000","last_active":"2024-03-27T08:41:05.586188+0000","last_peered":"2024-03-27T08:41:05.586188+0000","last_clean":"2024-03-27T08:41:05.586188+0000","last_became_active":"2024-03-27T08:27:58.967369+0000","last_became_peered":"2024-03-27T08:27:58.967369+0000","last_unstale":"2024-03-27T08:41:05.586188+0000","last_undegraded":"2024-03-27T08:41:05.586188+0000","last_fullsized":"2024-03-27T08:41:05.586188+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.945771+0000","last_change":"2024-03-27T08:27:58.970092+0000","last_active":"2024-03-27T08:41:03.945771+0000","last_peered":"2024-03-27T08:41:03.945771+0000","last_clean":"2024-03-27T08:41:03.945771+0000","last_became_active":"2024-03-27T08:27:58.956201+0000","last_became_peered":"2024-03-27T08:27:58.956201+0000","last_unstale":"2024-03-27T08:41:03.945771+0000","last_undegraded":"2024-03-27T08:41:03.945771+0000","last_fullsized":"2024-03-27T08:41:03.945771+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.925057+0000","last_change":"2024-03-27T08:27:59.003207+0000","last_active":"2024-03-27T08:41:03.925057+0000","last_peered":"2024-03-27T08:41:03.925057+0000","last_clean":"2024-03-27T08:41:03.925057+0000","last_became_active":"2024-03-27T08:27:58.981709+0000","last_became_peered":"2024-03-27T08:27:58.981709+0000","last_unstale":"2024-03-27T08:41:03.925057+0000","last_undegraded":"2024-03-27T08:41:03.925057+0000","last_fullsized":"2024-03-27T08:41:03.925057+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.903633+0000","last_change":"2024-03-27T08:27:58.997466+0000","last_active":"2024-03-27T08:41:03.903633+0000","last_peered":"2024-03-27T08:41:03.903633+0000","last_clean":"2024-03-27T08:41:03.903633+0000","last_became_active":"2024-03-27T08:27:58.977694+0000","last_became_peered":"2024-03-27T08:27:58.977694+0000","last_unstale":"2024-03-27T08:41:03.903633+0000","last_undegraded":"2024-03-27T08:41:03.903633+0000","last_fullsized":"2024-03-27T08:41:03.903633+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.644832+0000","last_change":"2024-03-27T08:27:59.001487+0000","last_active":"2024-03-27T08:41:05.644832+0000","last_peered":"2024-03-27T08:41:05.644832+0000","last_clean":"2024-03-27T08:41:05.644832+0000","last_became_active":"2024-03-27T08:27:58.979333+0000","last_became_peered":"2024-03-27T08:27:58.979333+0000","last_unstale":"2024-03-27T08:41:05.644832+0000","last_undegraded":"2024-03-27T08:41:05.644832+0000","last_fullsized":"2024-03-27T08:41:05.644832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.588534+0000","last_change":"2024-03-27T08:27:58.954275+0000","last_active":"2024-03-27T08:41:05.588534+0000","last_peered":"2024-03-27T08:41:05.588534+0000","last_clean":"2024-03-27T08:41:05.588534+0000","last_became_active":"2024-03-27T08:27:58.939180+0000","last_became_peered":"2024-03-27T08:27:58.939180+0000","last_unstale":"2024-03-27T08:41:05.588534+0000","last_undegraded":"2024-03-27T08:41:05.588534+0000","last_fullsized":"2024-03-27T08:41:05.588534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.658045+0000","last_change":"2024-03-27T08:27:58.988220+0000","last_active":"2024-03-27T08:41:05.658045+0000","last_peered":"2024-03-27T08:41:05.658045+0000","last_clean":"2024-03-27T08:41:05.658045+0000","last_became_active":"2024-03-27T08:27:58.955512+0000","last_became_peered":"2024-03-27T08:27:58.955512+0000","last_unstale":"2024-03-27T08:41:05.658045+0000","last_undegraded":"2024-03-27T08:41:05.658045+0000","last_fullsized":"2024-03-27T08:41:05.658045+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.916735+0000","last_change":"2024-03-27T08:27:59.103582+0000","last_active":"2024-03-27T08:41:03.916735+0000","last_peered":"2024-03-27T08:41:03.916735+0000","last_clean":"2024-03-27T08:41:03.916735+0000","last_became_active":"2024-03-27T08:27:59.098593+0000","last_became_peered":"2024-03-27T08:27:59.098593+0000","last_unstale":"2024-03-27T08:41:03.916735+0000","last_undegraded":"2024-03-27T08:41:03.916735+0000","last_fullsized":"2024-03-27T08:41:03.916735+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.693701+0000","last_change":"2024-03-27T08:27:58.999586+0000","last_active":"2024-03-27T08:41:05.693701+0000","last_peered":"2024-03-27T08:41:05.693701+0000","last_clean":"2024-03-27T08:41:05.693701+0000","last_became_active":"2024-03-27T08:27:58.973352+0000","last_became_peered":"2024-03-27T08:27:58.973352+0000","last_unstale":"2024-03-27T08:41:05.693701+0000","last_undegraded":"2024-03-27T08:41:05.693701+0000","last_fullsized":"2024-03-27T08:41:05.693701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.690909+0000","last_change":"2024-03-27T08:27:59.070788+0000","last_active":"2024-03-27T08:41:05.690909+0000","last_peered":"2024-03-27T08:41:05.690909+0000","last_clean":"2024-03-27T08:41:05.690909+0000","last_became_active":"2024-03-27T08:27:59.065991+0000","last_became_peered":"2024-03-27T08:27:59.065991+0000","last_unstale":"2024-03-27T08:41:05.690909+0000","last_undegraded":"2024-03-27T08:41:05.690909+0000","last_fullsized":"2024-03-27T08:41:05.690909+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.908630+0000","last_change":"2024-03-27T08:27:59.032231+0000","last_active":"2024-03-27T08:41:03.908630+0000","last_peered":"2024-03-27T08:41:03.908630+0000","last_clean":"2024-03-27T08:41:03.908630+0000","last_became_active":"2024-03-27T08:27:59.007454+0000","last_became_peered":"2024-03-27T08:27:59.007454+0000","last_unstale":"2024-03-27T08:41:03.908630+0000","last_undegraded":"2024-03-27T08:41:03.908630+0000","last_fullsized":"2024-03-27T08:41:03.908630+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.911667+0000","last_change":"2024-03-27T08:27:59.036310+0000","last_active":"2024-03-27T08:41:03.911667+0000","last_peered":"2024-03-27T08:41:03.911667+0000","last_clean":"2024-03-27T08:41:03.911667+0000","last_became_active":"2024-03-27T08:27:59.015211+0000","last_became_peered":"2024-03-27T08:27:59.015211+0000","last_unstale":"2024-03-27T08:41:03.911667+0000","last_undegraded":"2024-03-27T08:41:03.911667+0000","last_fullsized":"2024-03-27T08:41:03.911667+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.913897+0000","last_change":"2024-03-27T08:27:58.973584+0000","last_active":"2024-03-27T08:41:03.913897+0000","last_peered":"2024-03-27T08:41:03.913897+0000","last_clean":"2024-03-27T08:41:03.913897+0000","last_became_active":"2024-03-27T08:27:58.960673+0000","last_became_peered":"2024-03-27T08:27:58.960673+0000","last_unstale":"2024-03-27T08:41:03.913897+0000","last_undegraded":"2024-03-27T08:41:03.913897+0000","last_fullsized":"2024-03-27T08:41:03.913897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.701323+0000","last_change":"2024-03-27T08:27:59.040676+0000","last_active":"2024-03-27T08:41:05.701323+0000","last_peered":"2024-03-27T08:41:05.701323+0000","last_clean":"2024-03-27T08:41:05.701323+0000","last_became_active":"2024-03-27T08:27:59.034744+0000","last_became_peered":"2024-03-27T08:27:59.034744+0000","last_unstale":"2024-03-27T08:41:05.701323+0000","last_undegraded":"2024-03-27T08:41:05.701323+0000","last_fullsized":"2024-03-27T08:41:05.701323+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.623255+0000","last_change":"2024-03-27T08:27:58.993514+0000","last_active":"2024-03-27T08:41:05.623255+0000","last_peered":"2024-03-27T08:41:05.623255+0000","last_clean":"2024-03-27T08:41:05.623255+0000","last_became_active":"2024-03-27T08:27:58.988938+0000","last_became_peered":"2024-03-27T08:27:58.988938+0000","last_unstale":"2024-03-27T08:41:05.623255+0000","last_undegraded":"2024-03-27T08:41:05.623255+0000","last_fullsized":"2024-03-27T08:41:05.623255+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.637259+0000","last_change":"2024-03-27T08:27:58.953456+0000","last_active":"2024-03-27T08:41:05.637259+0000","last_peered":"2024-03-27T08:41:05.637259+0000","last_clean":"2024-03-27T08:41:05.637259+0000","last_became_active":"2024-03-27T08:27:58.940428+0000","last_became_peered":"2024-03-27T08:27:58.940428+0000","last_unstale":"2024-03-27T08:41:05.637259+0000","last_undegraded":"2024-03-27T08:41:05.637259+0000","last_fullsized":"2024-03-27T08:41:05.637259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.617111+0000","last_change":"2024-03-27T08:27:58.991121+0000","last_active":"2024-03-27T08:41:05.617111+0000","last_peered":"2024-03-27T08:41:05.617111+0000","last_clean":"2024-03-27T08:41:05.617111+0000","last_became_active":"2024-03-27T08:27:58.982971+0000","last_became_peered":"2024-03-27T08:27:58.982971+0000","last_unstale":"2024-03-27T08:41:05.617111+0000","last_undegraded":"2024-03-27T08:41:05.617111+0000","last_fullsized":"2024-03-27T08:41:05.617111+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1270,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.621935+0000","last_change":"2024-03-27T08:27:59.023638+0000","last_active":"2024-03-27T08:41:05.621935+0000","last_peered":"2024-03-27T08:41:05.621935+0000","last_clean":"2024-03-27T08:41:05.621935+0000","last_became_active":"2024-03-27T08:27:59.017861+0000","last_became_peered":"2024-03-27T08:27:59.017861+0000","last_unstale":"2024-03-27T08:41:05.621935+0000","last_undegraded":"2024-03-27T08:41:05.621935+0000","last_fullsized":"2024-03-27T08:41:05.621935+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.587067+0000","last_change":"2024-03-27T08:27:58.950095+0000","last_active":"2024-03-27T08:41:05.587067+0000","last_peered":"2024-03-27T08:41:05.587067+0000","last_clean":"2024-03-27T08:41:05.587067+0000","last_became_active":"2024-03-27T08:27:58.942454+0000","last_became_peered":"2024-03-27T08:27:58.942454+0000","last_unstale":"2024-03-27T08:41:05.587067+0000","last_undegraded":"2024-03-27T08:41:05.587067+0000","last_fullsized":"2024-03-27T08:41:05.587067+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.652353+0000","last_change":"2024-03-27T08:27:58.986152+0000","last_active":"2024-03-27T08:41:05.652353+0000","last_peered":"2024-03-27T08:41:05.652353+0000","last_clean":"2024-03-27T08:41:05.652353+0000","last_became_active":"2024-03-27T08:27:58.960185+0000","last_became_peered":"2024-03-27T08:27:58.960185+0000","last_unstale":"2024-03-27T08:41:05.652353+0000","last_undegraded":"2024-03-27T08:41:05.652353+0000","last_fullsized":"2024-03-27T08:41:05.652353+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.698522+0000","last_change":"2024-03-27T08:27:58.996597+0000","last_active":"2024-03-27T08:41:05.698522+0000","last_peered":"2024-03-27T08:41:05.698522+0000","last_clean":"2024-03-27T08:41:05.698522+0000","last_became_active":"2024-03-27T08:27:58.969756+0000","last_became_peered":"2024-03-27T08:27:58.969756+0000","last_unstale":"2024-03-27T08:41:05.698522+0000","last_undegraded":"2024-03-27T08:41:05.698522+0000","last_fullsized":"2024-03-27T08:41:05.698522+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.936336+0000","last_change":"2024-03-27T08:27:59.018353+0000","last_active":"2024-03-27T08:41:03.936336+0000","last_peered":"2024-03-27T08:41:03.936336+0000","last_clean":"2024-03-27T08:41:03.936336+0000","last_became_active":"2024-03-27T08:27:59.000404+0000","last_became_peered":"2024-03-27T08:27:59.000404+0000","last_unstale":"2024-03-27T08:41:03.936336+0000","last_undegraded":"2024-03-27T08:41:03.936336+0000","last_fullsized":"2024-03-27T08:41:03.936336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1250,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.667330+0000","last_change":"2024-03-27T08:27:59.012490+0000","last_active":"2024-03-27T08:41:05.667330+0000","last_peered":"2024-03-27T08:41:05.667330+0000","last_clean":"2024-03-27T08:41:05.667330+0000","last_became_active":"2024-03-27T08:27:59.004002+0000","last_became_peered":"2024-03-27T08:27:59.004002+0000","last_unstale":"2024-03-27T08:41:05.667330+0000","last_undegraded":"2024-03-27T08:41:05.667330+0000","last_fullsized":"2024-03-27T08:41:05.667330+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.922123+0000","last_change":"2024-03-27T08:27:59.052618+0000","last_active":"2024-03-27T08:41:03.922123+0000","last_peered":"2024-03-27T08:41:03.922123+0000","last_clean":"2024-03-27T08:41:03.922123+0000","last_became_active":"2024-03-27T08:27:59.045303+0000","last_became_peered":"2024-03-27T08:27:59.045303+0000","last_unstale":"2024-03-27T08:41:03.922123+0000","last_undegraded":"2024-03-27T08:41:03.922123+0000","last_fullsized":"2024-03-27T08:41:03.922123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.950979+0000","last_change":"2024-03-27T08:27:58.988986+0000","last_active":"2024-03-27T08:41:03.950979+0000","last_peered":"2024-03-27T08:41:03.950979+0000","last_clean":"2024-03-27T08:41:03.950979+0000","last_became_active":"2024-03-27T08:27:58.964911+0000","last_became_peered":"2024-03-27T08:27:58.964911+0000","last_unstale":"2024-03-27T08:41:03.950979+0000","last_undegraded":"2024-03-27T08:41:03.950979+0000","last_fullsized":"2024-03-27T08:41:03.950979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1286,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.941336+0000","last_change":"2024-03-27T08:27:59.011613+0000","last_active":"2024-03-27T08:41:03.941336+0000","last_peered":"2024-03-27T08:41:03.941336+0000","last_clean":"2024-03-27T08:41:03.941336+0000","last_became_active":"2024-03-27T08:27:58.985998+0000","last_became_peered":"2024-03-27T08:27:58.985998+0000","last_unstale":"2024-03-27T08:41:03.941336+0000","last_undegraded":"2024-03-27T08:41:03.941336+0000","last_fullsized":"2024-03-27T08:41:03.941336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1283,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.613949+0000","last_change":"2024-03-27T08:27:58.951044+0000","last_active":"2024-03-27T08:41:05.613949+0000","last_peered":"2024-03-27T08:41:05.613949+0000","last_clean":"2024-03-27T08:41:05.613949+0000","last_became_active":"2024-03-27T08:27:58.934959+0000","last_became_peered":"2024-03-27T08:27:58.934959+0000","last_unstale":"2024-03-27T08:41:05.613949+0000","last_undegraded":"2024-03-27T08:41:05.613949+0000","last_fullsized":"2024-03-27T08:41:05.613949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1256,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.611729+0000","last_change":"2024-03-27T08:27:58.978772+0000","last_active":"2024-03-27T08:41:05.611729+0000","last_peered":"2024-03-27T08:41:05.611729+0000","last_clean":"2024-03-27T08:41:05.611729+0000","last_became_active":"2024-03-27T08:27:58.972299+0000","last_became_peered":"2024-03-27T08:27:58.972299+0000","last_unstale":"2024-03-27T08:41:05.611729+0000","last_undegraded":"2024-03-27T08:41:05.611729+0000","last_fullsized":"2024-03-27T08:41:05.611729+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.612790+0000","last_change":"2024-03-27T08:23:09.066891+0000","last_active":"2024-03-27T08:41:05.612790+0000","last_peered":"2024-03-27T08:41:05.612790+0000","last_clean":"2024-03-27T08:41:05.612790+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:41:05.612790+0000","last_undegraded":"2024-03-27T08:41:05.612790+0000","last_fullsized":"2024-03-27T08:41:05.612790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1845,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.733759+0000","last_change":"2024-03-27T08:23:09.059885+0000","last_active":"2024-03-27T08:41:05.733759+0000","last_peered":"2024-03-27T08:41:05.733759+0000","last_clean":"2024-03-27T08:41:05.733759+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:41:05.733759+0000","last_undegraded":"2024-03-27T08:41:05.733759+0000","last_fullsized":"2024-03-27T08:41:05.733759+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1851,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.774666+0000","last_change":"2024-03-27T08:23:09.052784+0000","last_active":"2024-03-27T08:41:05.774666+0000","last_peered":"2024-03-27T08:41:05.774666+0000","last_clean":"2024-03-27T08:41:05.774666+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:41:05.774666+0000","last_undegraded":"2024-03-27T08:41:05.774666+0000","last_fullsized":"2024-03-27T08:41:05.774666+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1845,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.619459+0000","last_change":"2024-03-27T08:23:09.053884+0000","last_active":"2024-03-27T08:41:05.619459+0000","last_peered":"2024-03-27T08:41:05.619459+0000","last_clean":"2024-03-27T08:41:05.619459+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:41:05.619459+0000","last_undegraded":"2024-03-27T08:41:05.619459+0000","last_fullsized":"2024-03-27T08:41:05.619459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1887,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.976937+0000","last_change":"2024-03-27T08:23:09.061623+0000","last_active":"2024-03-27T08:41:03.976937+0000","last_peered":"2024-03-27T08:41:03.976937+0000","last_clean":"2024-03-27T08:41:03.976937+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:41:03.976937+0000","last_undegraded":"2024-03-27T08:41:03.976937+0000","last_fullsized":"2024-03-27T08:41:03.976937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.622848+0000","last_change":"2024-03-27T08:23:09.055461+0000","last_active":"2024-03-27T08:41:05.622848+0000","last_peered":"2024-03-27T08:41:05.622848+0000","last_clean":"2024-03-27T08:41:05.622848+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:41:05.622848+0000","last_undegraded":"2024-03-27T08:41:05.622848+0000","last_fullsized":"2024-03-27T08:41:05.622848+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1887,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:03.996706+0000","last_change":"2024-03-27T08:23:09.074614+0000","last_active":"2024-03-27T08:41:03.996706+0000","last_peered":"2024-03-27T08:41:03.996706+0000","last_clean":"2024-03-27T08:41:03.996706+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:41:03.996706+0000","last_undegraded":"2024-03-27T08:41:03.996706+0000","last_fullsized":"2024-03-27T08:41:03.996706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1859,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.688246+0000","last_change":"2024-03-27T08:23:09.065607+0000","last_active":"2024-03-27T08:41:05.688246+0000","last_peered":"2024-03-27T08:41:05.688246+0000","last_clean":"2024-03-27T08:41:05.688246+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:41:05.688246+0000","last_undegraded":"2024-03-27T08:41:05.688246+0000","last_fullsized":"2024-03-27T08:41:05.688246+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1984,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-27T08:41:05.692000+0000","last_change":"2024-03-27T08:23:07.031764+0000","last_active":"2024-03-27T08:41:05.692000+0000","last_peered":"2024-03-27T08:41:05.692000+0000","last_clean":"2024-03-27T08:41:05.692000+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:41:05.692000+0000","last_undegraded":"2024-03-27T08:41:05.692000+0000","last_fullsized":"2024-03-27T08:41:05.692000+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79604,"kb_used_data":4076,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104777996,"statfs":{"total":107374182400,"available":107292667904,"internally_reserved":0,"allocated":4173824,"data_stored":2026406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83260,"kb_used_data":3572,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104774340,"statfs":{"total":107374182400,"available":107288924160,"internally_reserved":0,"allocated":3657728,"data_stored":1464751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66568,"kb_used_data":3520,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791032,"statfs":{"total":107374182400,"available":107306016768,"internally_reserved":0,"allocated":3604480,"data_stored":1458999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83804,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773796,"statfs":{"total":107374182400,"available":107288367104,"internally_reserved":0,"allocated":4214784,"data_stored":2061415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:41:37.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T08:41:37.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.595+0000 7fa104df5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:37.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.599+0000 7fa104df5640 1 Processor -- start 2024-03-27T08:41:37.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.599+0000 7fa104df5640 1 -- start start 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa104df5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa10009b250 0x7fa10009b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa104df5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa100075160 con 0x7fa1001324f0 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa104df5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa1000685f0 con 0x7fa10009b250 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fe59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa10009b250 0x7fa10009b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fdd9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa1001324f0 0x7fa100132900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57478/0 (socket says 172.21.15.97:57478) 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fe59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa10009b250 0x7fa10009b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52540/0 (socket says 172.21.15.97:52540) 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fdd9b640 1 -- 172.21.15.97:0/668110180 learned_addr learned my addr 172.21.15.97:0/668110180 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fe59c640 1 -- 172.21.15.97:0/668110180 >> v1:172.21.15.97:6789/0 conn(0x7fa1001324f0 legacy=0x7fa100132900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fd59a640 1 -- 172.21.15.97:0/668110180 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 221162302 0 0) 0x7fa100075160 con 0x7fa1001324f0 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fe59c640 1 -- 172.21.15.97:0/668110180 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1000709c0 con 0x7fa10009b250 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fe59c640 1 --2- 172.21.15.97:0/668110180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10009b660 crc :-1 s=READY pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1039cc4046d5a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fd59a640 1 -- 172.21.15.97:0/668110180 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0f000e040 con 0x7fa10009b250 2024-03-27T08:41:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fd59a640 1 -- 172.21.15.97:0/668110180 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa0f0007670 con 0x7fa10009b250 2024-03-27T08:41:37.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.601+0000 7fa0fd59a640 1 -- 172.21.15.97:0/668110180 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0f0007910 con 0x7fa10009b250 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 -- 172.21.15.97:0/668110180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 msgr2=0x7fa10009b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 --2- 172.21.15.97:0/668110180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10009b660 crc :-1 s=READY pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 -- 172.21.15.97:0/668110180 shutdown_connections 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 --2- 172.21.15.97:0/668110180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10009b660 unknown :-1 s=CLOSED pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 -- 172.21.15.97:0/668110180 >> 172.21.15.97:0/668110180 conn(0x7fa1000a0c40 msgr2=0x7fa1000a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 -- 172.21.15.97:0/668110180 shutdown_connections 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 1 -- 172.21.15.97:0/668110180 wait complete. 2024-03-27T08:41:37.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.602+0000 7fa104df5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.604+0000 7fa104df5640 1 Processor -- start 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.604+0000 7fa104df5640 1 -- start start 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.605+0000 7fa104df5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10011fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.605+0000 7fa104df5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa100122350 con 0x7fa10009b250 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.605+0000 7fa0fe59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10011fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.605+0000 7fa0fe59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10011fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52542/0 (socket says 172.21.15.97:52542) 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.605+0000 7fa0fe59c640 1 -- 172.21.15.97:0/43451637 learned_addr learned my addr 172.21.15.97:0/43451637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.605+0000 7fa0fe59c640 1 -- 172.21.15.97:0/43451637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100121a70 con 0x7fa10009b250 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.606+0000 7fa0fe59c640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10011fef0 crc :-1 s=READY pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.606+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0f000e040 con 0x7fa10009b250 2024-03-27T08:41:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.606+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa0f0013aa0 con 0x7fa10009b250 2024-03-27T08:41:37.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.606+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa100121010 con 0x7fa10009b250 2024-03-27T08:41:37.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.606+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa0f0013d40 con 0x7fa10009b250 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.606+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa100122b70 con 0x7fa10009b250 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.607+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa100123e50 con 0x7fa10009b250 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.607+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fa0f0011040 con 0x7fa10009b250 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.607+0000 7fa0eeffd640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa0e0038750 0x7fa0e003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.607+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 399..714) ==== 3403+0+0 (crc 0 0 0) 0x7fa0f0049b70 con 0x7fa10009b250 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.607+0000 7fa0fdd9b640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa0e0038750 0x7fa0e003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:37.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.608+0000 7fa0fdd9b640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa0e0038750 0x7fa0e003ac60 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:37.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.612+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fa0f0016e40 con 0x7fa10009b250 2024-03-27T08:41:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.788+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa100063450 con 0x7fa10009b250 2024-03-27T08:41:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.789+0000 7fa0eeffd640 1 -- 172.21.15.97:0/43451637 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v714) ==== 74+0+11127 (crc 0 0 0) 0x7fa0f00112f0 con 0x7fa10009b250 2024-03-27T08:41:37.782 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:41:37.782 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":714,"fsid":"ab3c95de-2422-462d-839c-7edbd2518ef5","created":"2024-03-27T08:22:57.667418+0000","modified":"2024-03-27T08:41:35.866291+0000","last_up_change":"2024-03-27T08:23:03.975803+0000","last_in_change":"2024-03-27T08:22:59.784887+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":201,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T08:23:05.955717+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-27T08:23:07.988518+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":75,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-03-27T08:27:56.798305+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8aee0ca6-e299-4707-bb1a-9d92e6da31c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":701,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":3203341735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3203341735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3203341735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3203341735}]},"public_addr":"172.21.15.97:6801/3203341735","cluster_addr":"172.21.15.97:6802/3203341735","heartbeat_back_addr":"172.21.15.97:6812/3203341735","heartbeat_front_addr":"172.21.15.97:6811/3203341735","state":["exists","up"]},{"osd":1,"uuid":"ea23b051-821d-4838-b8ef-1cb78fa81d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":701,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":1668842645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1668842645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1668842645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":1668842645}]},"public_addr":"172.21.15.97:6805/1668842645","cluster_addr":"172.21.15.97:6804/1668842645","heartbeat_back_addr":"172.21.15.97:6806/1668842645","heartbeat_front_addr":"172.21.15.97:6803/1668842645","state":["exists","up"]},{"osd":2,"uuid":"b1d0dc27-69b7-42ad-99fb-50b882a4bfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":701,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":239686547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":239686547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":239686547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6807","nonce":239686547}]},"public_addr":"172.21.15.190:6809/239686547","cluster_addr":"172.21.15.190:6808/239686547","heartbeat_back_addr":"172.21.15.190:6810/239686547","heartbeat_front_addr":"172.21.15.190:6807/239686547","state":["exists","up"]},{"osd":3,"uuid":"c15879d9-12a6-4558-a5ed-1bc46b9961a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":701,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":971408905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":971408905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6803","nonce":971408905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":971408905}]},"public_addr":"172.21.15.190:6805/971408905","cluster_addr":"172.21.15.190:6804/971408905","heartbeat_back_addr":"172.21.15.190:6803/971408905","heartbeat_front_addr":"172.21.15.190:6802/971408905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":100,"snaps":[{"begin":1,"length":4}]},{"pool":101,"snaps":[{"begin":2,"length":4},{"begin":7,"length":2}]},{"pool":102,"snaps":[{"begin":1,"length":6}]},{"pool":103,"snaps":[{"begin":2,"length":4},{"begin":9,"length":1}]},{"pool":104,"snaps":[{"begin":2,"length":1}]},{"pool":180,"snaps":[{"begin":1,"length":1}]},{"pool":181,"snaps":[{"begin":1,"length":1}]},{"pool":183,"snaps":[{"begin":1,"length":1}]},{"pool":184,"snaps":[{"begin":1,"length":2}]},{"pool":185,"snaps":[{"begin":3,"length":1}]},{"pool":188,"snaps":[{"begin":2,"length":2}]},{"pool":190,"snaps":[{"begin":3,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa0e0038750 msgr2=0x7fa0e003ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa0e0038750 0x7fa0e003ac60 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 msgr2=0x7fa10011fef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10011fef0 crc :-1 s=READY pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 shutdown_connections 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fa0e0038750 0x7fa0e003ac60 unknown :-1 s=CLOSED pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 --2- 172.21.15.97:0/43451637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa10009b250 0x7fa10011fef0 unknown :-1 s=CLOSED pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 >> 172.21.15.97:0/43451637 conn(0x7fa1000a0c40 msgr2=0x7fa1001191f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 shutdown_connections 2024-03-27T08:41:37.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:37.791+0000 7fa104df5640 1 -- 172.21.15.97:0/43451637 wait complete. 2024-03-27T08:41:38.795 INFO:tasks.ceph:Scrubbing osd.0 2024-03-27T08:41:38.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:41:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.912+0000 7f7a78aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:38.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.921+0000 7f7a78aed640 1 Processor -- start 2024-03-27T08:41:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.921+0000 7f7a78aed640 1 -- start start 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a74133cc0 0x7f7a741360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a7406f2c0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7a74130950 con 0x7f7a74133cc0 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a7259c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7a7406e4f0 0x7f7a74130f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38958/0 (socket says 172.21.15.97:38958) 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a7259c640 1 -- 172.21.15.97:0/2324832343 learned_addr learned my addr 172.21.15.97:0/2324832343 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a71d9b640 1 --2- 172.21.15.97:0/2324832343 >> v2:172.21.15.97:3300/0 conn(0x7f7a74133cc0 0x7f7a741360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a71d9b640 1 -- 172.21.15.97:0/2324832343 >> v1:172.21.15.97:6789/0 conn(0x7f7a7406e4f0 legacy=0x7f7a74130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a71d9b640 1 -- 172.21.15.97:0/2324832343 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a74130c60 con 0x7f7a74133cc0 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a71d9b640 1 --2- 172.21.15.97:0/2324832343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a74133cc0 0x7f7a741360e0 crc :-1 s=READY pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f9534979b47f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a7159a640 1 -- 172.21.15.97:0/2324832343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7a64009d90 con 0x7f7a74133cc0 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a7159a640 1 -- 172.21.15.97:0/2324832343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7a6400f340 con 0x7f7a74133cc0 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a7159a640 1 -- 172.21.15.97:0/2324832343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7a6400f5e0 con 0x7f7a74133cc0 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- 172.21.15.97:0/2324832343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a74133cc0 msgr2=0x7f7a741360e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 --2- 172.21.15.97:0/2324832343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a74133cc0 0x7f7a741360e0 crc :-1 s=READY pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- 172.21.15.97:0/2324832343 shutdown_connections 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 --2- 172.21.15.97:0/2324832343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a74133cc0 0x7f7a741360e0 unknown :-1 s=CLOSED pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- 172.21.15.97:0/2324832343 >> 172.21.15.97:0/2324832343 conn(0x7f7a7409d1d0 msgr2=0x7f7a7409d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- 172.21.15.97:0/2324832343 shutdown_connections 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.923+0000 7f7a78aed640 1 -- 172.21.15.97:0/2324832343 wait complete. 2024-03-27T08:41:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.924+0000 7f7a78aed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.925+0000 7f7a78aed640 1 Processor -- start 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.925+0000 7f7a78aed640 1 -- start start 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.926+0000 7f7a78aed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 0x7f7a741c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.926+0000 7f7a78aed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7412ad90 con 0x7f7a7406e4f0 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.926+0000 7f7a7259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 0x7f7a741c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.926+0000 7f7a7259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 0x7f7a741c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60748/0 (socket says 172.21.15.97:60748) 2024-03-27T08:41:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.926+0000 7f7a7259c640 1 -- 172.21.15.97:0/55610771 learned_addr learned my addr 172.21.15.97:0/55610771 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.926+0000 7f7a7259c640 1 -- 172.21.15.97:0/55610771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7406a9e0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.927+0000 7f7a7259c640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 0x7f7a741c43b0 crc :-1 s=READY pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.927+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7a6c0070b0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.927+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7a6c007210 con 0x7f7a7406e4f0 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.927+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7411c290 con 0x7f7a7406e4f0 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.927+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7a6c0074b0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.927+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7411c900 con 0x7f7a7406e4f0 2024-03-27T08:41:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a7411d8f0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f7a6c0076a0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a62ffd640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7a540387a0 0x7f7a5403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f7a6c048c50 con 0x7f7a7406e4f0 2024-03-27T08:41:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a62ffd640 1 --2- 172.21.15.97:0/55610771 >> v2:172.21.15.97:6801/3203341735 conn(0x7f7a5403e860 0x7f7a54040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 --> v2:172.21.15.97:6801/3203341735 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a540413e0 con 0x7f7a5403e860 2024-03-27T08:41:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=714) ==== 24+0+0 (crc 0 0 0) 0x7f7a6c048fe0 con 0x7f7a7406e4f0 2024-03-27T08:41:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.928+0000 7f7a71d9b640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7a540387a0 0x7f7a5403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.929+0000 7f7a72d9d640 1 --2- 172.21.15.97:0/55610771 >> v2:172.21.15.97:6801/3203341735 conn(0x7f7a5403e860 0x7f7a54040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.929+0000 7f7a71d9b640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7a540387a0 0x7f7a5403acb0 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.931+0000 7f7a72d9d640 1 --2- 172.21.15.97:0/55610771 >> v2:172.21.15.97:6801/3203341735 conn(0x7f7a5403e860 0x7f7a54040d50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.932+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== osd.0 v2:172.21.15.97:6801/3203341735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f7a540413e0 con 0x7f7a5403e860 2024-03-27T08:41:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.936+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 --> v2:172.21.15.97:6801/3203341735 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a7409cd50 con 0x7f7a5403e860 2024-03-27T08:41:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a62ffd640 1 -- 172.21.15.97:0/55610771 <== osd.0 v2:172.21.15.97:6801/3203341735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f7a68009830 con 0x7f7a5403e860 2024-03-27T08:41:38.931 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T08:41:38.931 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "" 2024-03-27T08:41:38.932 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 >> v2:172.21.15.97:6801/3203341735 conn(0x7f7a5403e860 msgr2=0x7f7a54040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 --2- 172.21.15.97:0/55610771 >> v2:172.21.15.97:6801/3203341735 conn(0x7f7a5403e860 0x7f7a54040d50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7a540387a0 msgr2=0x7f7a5403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7a540387a0 0x7f7a5403acb0 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 msgr2=0x7f7a741c43b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 0x7f7a741c43b0 crc :-1 s=READY pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 shutdown_connections 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 --2- 172.21.15.97:0/55610771 >> v2:172.21.15.97:6801/3203341735 conn(0x7f7a5403e860 0x7f7a54040d50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f7a540387a0 0x7f7a5403acb0 unknown :-1 s=CLOSED pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 --2- 172.21.15.97:0/55610771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7406e4f0 0x7f7a741c43b0 unknown :-1 s=CLOSED pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 >> 172.21.15.97:0/55610771 conn(0x7f7a7409d1d0 msgr2=0x7f7a7412f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:38.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 shutdown_connections 2024-03-27T08:41:38.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:38.938+0000 7f7a78aed640 1 -- 172.21.15.97:0/55610771 wait complete. 2024-03-27T08:41:38.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-27T08:41:39.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.056+0000 7f4d9382d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.060+0000 7f4d9382d640 1 Processor -- start 2024-03-27T08:41:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.061+0000 7f4d9382d640 1 -- start start 2024-03-27T08:41:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d9382d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d8c131650 0x7f4d8c131a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d9382d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c0a06a0 con 0x7f4d8c1345b0 2024-03-27T08:41:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d9382d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8c0a03e0 con 0x7f4d8c131650 2024-03-27T08:41:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d915c9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d8c131650 0x7f4d8c131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d90dc8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4d8c1345b0 0x7f4d8c1349c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38964/0 (socket says 172.21.15.97:38964) 2024-03-27T08:41:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d90dc8640 1 -- 172.21.15.97:0/2419164187 learned_addr learned my addr 172.21.15.97:0/2419164187 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d915c9640 1 -- 172.21.15.97:0/2419164187 >> v1:172.21.15.97:6789/0 conn(0x7f4d8c1345b0 legacy=0x7f4d8c1349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d915c9640 1 -- 172.21.15.97:0/2419164187 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c06de30 con 0x7f4d8c131650 2024-03-27T08:41:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d915c9640 1 --2- 172.21.15.97:0/2419164187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c131a60 crc :-1 s=READY pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f2c0e02ee066b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.062+0000 7f4d83fff640 1 -- 172.21.15.97:0/2419164187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d8800e040 con 0x7f4d8c131650 2024-03-27T08:41:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d83fff640 1 -- 172.21.15.97:0/2419164187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d88007670 con 0x7f4d8c131650 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d83fff640 1 -- 172.21.15.97:0/2419164187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d88007910 con 0x7f4d8c131650 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 -- 172.21.15.97:0/2419164187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 msgr2=0x7f4d8c131a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 --2- 172.21.15.97:0/2419164187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c131a60 crc :-1 s=READY pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 -- 172.21.15.97:0/2419164187 shutdown_connections 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 --2- 172.21.15.97:0/2419164187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c131a60 unknown :-1 s=CLOSED pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 -- 172.21.15.97:0/2419164187 >> 172.21.15.97:0/2419164187 conn(0x7f4d8c0a15e0 msgr2=0x7f4d8c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 -- 172.21.15.97:0/2419164187 shutdown_connections 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.063+0000 7f4d9382d640 1 -- 172.21.15.97:0/2419164187 wait complete. 2024-03-27T08:41:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.064+0000 7f4d9382d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.065+0000 7f4d9382d640 1 Processor -- start 2024-03-27T08:41:39.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.065+0000 7f4d9382d640 1 -- start start 2024-03-27T08:41:39.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d9382d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c120000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d9382d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c131200 con 0x7f4d8c131650 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d915c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c120000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d915c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c120000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60772/0 (socket says 172.21.15.97:60772) 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d915c9640 1 -- 172.21.15.97:0/821082011 learned_addr learned my addr 172.21.15.97:0/821082011 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d915c9640 1 -- 172.21.15.97:0/821082011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c120e20 con 0x7f4d8c131650 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d915c9640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c120000 crc :-1 s=READY pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.066+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d8800e040 con 0x7f4d8c131650 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.067+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d88012a90 con 0x7f4d8c131650 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.067+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c1208a0 con 0x7f4d8c131650 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.067+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d88012d30 con 0x7f4d8c131650 2024-03-27T08:41:39.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.067+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c11f870 con 0x7f4d8c131650 2024-03-27T08:41:39.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.067+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f4d880153f0 con 0x7f4d8c131650 2024-03-27T08:41:39.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.068+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8c120c20 con 0x7f4d8c131650 2024-03-27T08:41:39.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.068+0000 7f4d81ffb640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4d700387a0 0x7f4d7003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.068+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f4d88049ae0 con 0x7f4d8c131650 2024-03-27T08:41:39.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.068+0000 7f4d90dc8640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4d700387a0 0x7f4d7003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.069+0000 7f4d90dc8640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4d700387a0 0x7f4d7003acb0 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.074+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f4d8800fc80 con 0x7f4d8c131650 2024-03-27T08:41:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.376+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f4d8c11fb20 con 0x7f4d700387a0 2024-03-27T08:41:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.377+0000 7f4d81ffb640 1 -- 172.21.15.97:0/821082011 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f4d7c001250 con 0x7f4d700387a0 2024-03-27T08:41:39.368 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2024-03-27T08:41:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4d700387a0 msgr2=0x7f4d7003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4d700387a0 0x7f4d7003acb0 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 msgr2=0x7f4d8c120000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c120000 crc :-1 s=READY pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 shutdown_connections 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f4d700387a0 0x7f4d7003acb0 unknown :-1 s=CLOSED pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 --2- 172.21.15.97:0/821082011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c131650 0x7f4d8c120000 unknown :-1 s=CLOSED pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 >> 172.21.15.97:0/821082011 conn(0x7f4d8c0a15e0 msgr2=0x7f4d8c1bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 shutdown_connections 2024-03-27T08:41:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.379+0000 7f4d9382d640 1 -- 172.21.15.97:0/821082011 wait complete. 2024-03-27T08:41:39.385 INFO:tasks.ceph:Scrubbing osd.1 2024-03-27T08:41:39.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:41:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.505+0000 7f38d239d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.514+0000 7f38d239d640 1 Processor -- start 2024-03-27T08:41:39.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.514+0000 7f38d239d640 1 -- start start 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38d239d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38cc09f5f0 0x7f38cc09fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38d239d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0a3470 con 0x7f38cc09ff40 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38d239d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f38cc09f2e0 con 0x7f38cc09f5f0 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38cbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38cc09f5f0 0x7f38cc09fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38cb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38cc09ff40 0x7f38cc1345b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38966/0 (socket says 172.21.15.97:38966) 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38cb7fe640 1 -- 172.21.15.97:0/3817507376 learned_addr learned my addr 172.21.15.97:0/3817507376 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38cbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38cc09f5f0 0x7f38cc09fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60780/0 (socket says 172.21.15.97:60780) 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38cbfff640 1 -- 172.21.15.97:0/3817507376 >> v1:172.21.15.97:6789/0 conn(0x7f38cc09ff40 legacy=0x7f38cc1345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38cbfff640 1 -- 172.21.15.97:0/3817507376 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc06ea10 con 0x7f38cc09f5f0 2024-03-27T08:41:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.515+0000 7f38caffd640 1 -- 172.21.15.97:0/3817507376 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623052036 0 0) 0x7f38cc0a3470 con 0x7f38cc09ff40 2024-03-27T08:41:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38cbfff640 1 --2- 172.21.15.97:0/3817507376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc09fa00 crc :-1 s=READY pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32d6e9bcabf3633 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38caffd640 1 -- 172.21.15.97:0/3817507376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f38bc009d90 con 0x7f38cc09f5f0 2024-03-27T08:41:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38caffd640 1 -- 172.21.15.97:0/3817507376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f38bc00e230 con 0x7f38cc09f5f0 2024-03-27T08:41:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38caffd640 1 -- 172.21.15.97:0/3817507376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f38bc00e4d0 con 0x7f38cc09f5f0 2024-03-27T08:41:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38d239d640 1 -- 172.21.15.97:0/3817507376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 msgr2=0x7f38cc09fa00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38d239d640 1 --2- 172.21.15.97:0/3817507376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc09fa00 crc :-1 s=READY pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38d239d640 1 -- 172.21.15.97:0/3817507376 shutdown_connections 2024-03-27T08:41:39.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38d239d640 1 --2- 172.21.15.97:0/3817507376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc09fa00 unknown :-1 s=CLOSED pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.516+0000 7f38d239d640 1 -- 172.21.15.97:0/3817507376 >> 172.21.15.97:0/3817507376 conn(0x7f38cc0a15e0 msgr2=0x7f38cc0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.517+0000 7f38d239d640 1 -- 172.21.15.97:0/3817507376 shutdown_connections 2024-03-27T08:41:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.517+0000 7f38d239d640 1 -- 172.21.15.97:0/3817507376 wait complete. 2024-03-27T08:41:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.517+0000 7f38d239d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.518+0000 7f38d239d640 1 Processor -- start 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.518+0000 7f38d239d640 1 -- start start 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.519+0000 7f38d239d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc11dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.519+0000 7f38d239d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc11f090 con 0x7f38cc09f5f0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.519+0000 7f38cbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc11dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.519+0000 7f38cbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc11dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60784/0 (socket says 172.21.15.97:60784) 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.519+0000 7f38cbfff640 1 -- 172.21.15.97:0/3344877350 learned_addr learned my addr 172.21.15.97:0/3344877350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38cbfff640 1 -- 172.21.15.97:0/3344877350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc121360 con 0x7f38cc09f5f0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38cbfff640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc11dd90 crc :-1 s=READY pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f38bc013610 con 0x7f38cc09f5f0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f38bc013770 con 0x7f38cc09f5f0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc11f910 con 0x7f38cc09f5f0 2024-03-27T08:41:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f38bc013a10 con 0x7f38cc09f5f0 2024-03-27T08:41:39.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.520+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc06a9e0 con 0x7f38cc09f5f0 2024-03-27T08:41:39.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38cc134cc0 con 0x7f38cc09f5f0 2024-03-27T08:41:39.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f38bc013bc0 con 0x7f38cc09f5f0 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38c8ff9640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38b80387a0 0x7f38b803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f38bc018040 con 0x7f38cc09f5f0 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38c8ff9640 1 --2- 172.21.15.97:0/3344877350 >> v2:172.21.15.97:6805/1668842645 conn(0x7f38b803e860 0x7f38b8040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 --> v2:172.21.15.97:6805/1668842645 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38b80413e0 con 0x7f38b803e860 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=714) ==== 24+0+0 (crc 0 0 0) 0x7f38bc049d40 con 0x7f38cc09f5f0 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.521+0000 7f38cb7fe640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38b80387a0 0x7f38b803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.522+0000 7f38cb7fe640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38b80387a0 0x7f38b803acb0 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.522+0000 7f38d093a640 1 --2- 172.21.15.97:0/3344877350 >> v2:172.21.15.97:6805/1668842645 conn(0x7f38b803e860 0x7f38b8040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:39.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.524+0000 7f38d093a640 1 --2- 172.21.15.97:0/3344877350 >> v2:172.21.15.97:6805/1668842645 conn(0x7f38b803e860 0x7f38b8040d50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.526+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== osd.1 v2:172.21.15.97:6805/1668842645 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f38b80413e0 con 0x7f38b803e860 2024-03-27T08:41:39.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.529+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 --> v2:172.21.15.97:6805/1668842645 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f38cc121e00 con 0x7f38b803e860 2024-03-27T08:41:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.530+0000 7f38c8ff9640 1 -- 172.21.15.97:0/3344877350 <== osd.1 v2:172.21.15.97:6805/1668842645 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f38cc121e00 con 0x7f38b803e860 2024-03-27T08:41:39.521 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T08:41:39.522 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "" 2024-03-27T08:41:39.522 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 >> v2:172.21.15.97:6805/1668842645 conn(0x7f38b803e860 msgr2=0x7f38b8040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 --2- 172.21.15.97:0/3344877350 >> v2:172.21.15.97:6805/1668842645 conn(0x7f38b803e860 0x7f38b8040d50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38b80387a0 msgr2=0x7f38b803acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38b80387a0 0x7f38b803acb0 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 msgr2=0x7f38cc11dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc11dd90 crc :-1 s=READY pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 shutdown_connections 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 --2- 172.21.15.97:0/3344877350 >> v2:172.21.15.97:6805/1668842645 conn(0x7f38b803e860 0x7f38b8040d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f38b80387a0 0x7f38b803acb0 unknown :-1 s=CLOSED pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 --2- 172.21.15.97:0/3344877350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38cc09f5f0 0x7f38cc11dd90 unknown :-1 s=CLOSED pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 >> 172.21.15.97:0/3344877350 conn(0x7f38cc0a15e0 msgr2=0x7f38cc1bc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 shutdown_connections 2024-03-27T08:41:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.531+0000 7f38d239d640 1 -- 172.21.15.97:0/3344877350 wait complete. 2024-03-27T08:41:39.533 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-27T08:41:39.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.648+0000 7fdf46c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.654+0000 7fdf46c2d640 1 Processor -- start 2024-03-27T08:41:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.654+0000 7fdf46c2d640 1 -- start start 2024-03-27T08:41:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf46c2d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf40132000 0x7fdf40136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf46c2d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4006f950 con 0x7fdf401314e0 2024-03-27T08:41:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf46c2d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdf4006e6a0 con 0x7fdf40132000 2024-03-27T08:41:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf3ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf40132000 0x7fdf40136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf449c9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdf401314e0 0x7fdf401318f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38974/0 (socket says 172.21.15.97:38974) 2024-03-27T08:41:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf3ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf40132000 0x7fdf40136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60800/0 (socket says 172.21.15.97:60800) 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.655+0000 7fdf449c9640 1 -- 172.21.15.97:0/276525158 learned_addr learned my addr 172.21.15.97:0/276525158 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.656+0000 7fdf3ffff640 1 -- 172.21.15.97:0/276525158 >> v1:172.21.15.97:6789/0 conn(0x7fdf401314e0 legacy=0x7fdf401318f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.656+0000 7fdf3ffff640 1 -- 172.21.15.97:0/276525158 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf40075160 con 0x7fdf40132000 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.656+0000 7fdf3ffff640 1 --2- 172.21.15.97:0/276525158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40132000 0x7fdf40136a10 crc :-1 s=READY pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a522f88aa153541 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.656+0000 7fdf3f7fe640 1 -- 172.21.15.97:0/276525158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdf34009d90 con 0x7fdf40132000 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.656+0000 7fdf3f7fe640 1 -- 172.21.15.97:0/276525158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdf3400e230 con 0x7fdf40132000 2024-03-27T08:41:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.656+0000 7fdf3f7fe640 1 -- 172.21.15.97:0/276525158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdf3400e4d0 con 0x7fdf40132000 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 -- 172.21.15.97:0/276525158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40132000 msgr2=0x7fdf40136a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 --2- 172.21.15.97:0/276525158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40132000 0x7fdf40136a10 crc :-1 s=READY pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 -- 172.21.15.97:0/276525158 shutdown_connections 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 --2- 172.21.15.97:0/276525158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40132000 0x7fdf40136a10 unknown :-1 s=CLOSED pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 -- 172.21.15.97:0/276525158 >> 172.21.15.97:0/276525158 conn(0x7fdf400a0c40 msgr2=0x7fdf400a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 -- 172.21.15.97:0/276525158 shutdown_connections 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 1 -- 172.21.15.97:0/276525158 wait complete. 2024-03-27T08:41:39.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.657+0000 7fdf46c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.658+0000 7fdf46c2d640 1 Processor -- start 2024-03-27T08:41:39.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.658+0000 7fdf46c2d640 1 -- start start 2024-03-27T08:41:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.659+0000 7fdf46c2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 0x7fdf401c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.659+0000 7fdf46c2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4011e5e0 con 0x7fdf401314e0 2024-03-27T08:41:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf449c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 0x7fdf401c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf449c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 0x7fdf401c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60814/0 (socket says 172.21.15.97:60814) 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf449c9640 1 -- 172.21.15.97:0/1132041635 learned_addr learned my addr 172.21.15.97:0/1132041635 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf449c9640 1 -- 172.21.15.97:0/1132041635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4011edf0 con 0x7fdf401314e0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf449c9640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 0x7fdf401c6860 crc :-1 s=READY pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdf30005a60 con 0x7fdf401314e0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdf30005bc0 con 0x7fdf401314e0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdf30005e60 con 0x7fdf401314e0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4011eb50 con 0x7fdf401314e0 2024-03-27T08:41:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.660+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf40120090 con 0x7fdf401314e0 2024-03-27T08:41:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.661+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7fdf3000c780 con 0x7fdf401314e0 2024-03-27T08:41:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.661+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf40063450 con 0x7fdf401314e0 2024-03-27T08:41:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.661+0000 7fdf3d7fa640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdf2c0387a0 0x7fdf2c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.661+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7fdf30048180 con 0x7fdf401314e0 2024-03-27T08:41:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.662+0000 7fdf3ffff640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdf2c0387a0 0x7fdf2c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.662+0000 7fdf3ffff640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdf2c0387a0 0x7fdf2c03acb0 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.667+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7fdf3000d330 con 0x7fdf401314e0 2024-03-27T08:41:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.844+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fdf4009c9e0 con 0x7fdf2c0387a0 2024-03-27T08:41:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.845+0000 7fdf3d7fa640 1 -- 172.21.15.97:0/1132041635 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fdf34009e30 con 0x7fdf2c0387a0 2024-03-27T08:41:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2024-03-27T08:41:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdf2c0387a0 msgr2=0x7fdf2c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdf2c0387a0 0x7fdf2c03acb0 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 msgr2=0x7fdf401c6860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 0x7fdf401c6860 crc :-1 s=READY pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 shutdown_connections 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7fdf2c0387a0 0x7fdf2c03acb0 unknown :-1 s=CLOSED pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 --2- 172.21.15.97:0/1132041635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf401314e0 0x7fdf401c6860 unknown :-1 s=CLOSED pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 >> 172.21.15.97:0/1132041635 conn(0x7fdf400a0c40 msgr2=0x7fdf401b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 shutdown_connections 2024-03-27T08:41:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.848+0000 7fdf46c2d640 1 -- 172.21.15.97:0/1132041635 wait complete. 2024-03-27T08:41:39.853 INFO:tasks.ceph:Scrubbing osd.2 2024-03-27T08:41:39.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:41:39.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.969+0000 7f186039d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.978+0000 7f186039d640 1 Processor -- start 2024-03-27T08:41:39.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.978+0000 7f186039d640 1 -- start start 2024-03-27T08:41:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.979+0000 7f186039d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18581346a0 0x7f1858136ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.979+0000 7f186039d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185806fbb0 con 0x7f185806e4b0 2024-03-27T08:41:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.979+0000 7f186039d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f185809cf90 con 0x7f18581346a0 2024-03-27T08:41:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.979+0000 7f185e139640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f185806e4b0 0x7f185806e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38980/0 (socket says 172.21.15.97:38980) 2024-03-27T08:41:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.979+0000 7f185e139640 1 -- 172.21.15.97:0/3510149403 learned_addr learned my addr 172.21.15.97:0/3510149403 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.979+0000 7f185d938640 1 --2- 172.21.15.97:0/3510149403 >> v2:172.21.15.97:3300/0 conn(0x7f18581346a0 0x7f1858136ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.980+0000 7f185d938640 1 -- 172.21.15.97:0/3510149403 >> v1:172.21.15.97:6789/0 conn(0x7f185806e4b0 legacy=0x7f185806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.980+0000 7f185d938640 1 -- 172.21.15.97:0/3510149403 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858069270 con 0x7f18581346a0 2024-03-27T08:41:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.980+0000 7f185d938640 1 --2- 172.21.15.97:0/3510149403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18581346a0 0x7f1858136ac0 crc :-1 s=READY pgs=13025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acae9965c2fa0628 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.980+0000 7f185d137640 1 -- 172.21.15.97:0/3510149403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1848009d90 con 0x7f18581346a0 2024-03-27T08:41:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.980+0000 7f185d137640 1 -- 172.21.15.97:0/3510149403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f184800e230 con 0x7f18581346a0 2024-03-27T08:41:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.980+0000 7f185d137640 1 -- 172.21.15.97:0/3510149403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f184800e4d0 con 0x7f18581346a0 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 -- 172.21.15.97:0/3510149403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18581346a0 msgr2=0x7f1858136ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 --2- 172.21.15.97:0/3510149403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18581346a0 0x7f1858136ac0 crc :-1 s=READY pgs=13025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 -- 172.21.15.97:0/3510149403 shutdown_connections 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 --2- 172.21.15.97:0/3510149403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18581346a0 0x7f1858136ac0 unknown :-1 s=CLOSED pgs=13025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 -- 172.21.15.97:0/3510149403 >> 172.21.15.97:0/3510149403 conn(0x7f185809d1d0 msgr2=0x7f185809d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 -- 172.21.15.97:0/3510149403 shutdown_connections 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 1 -- 172.21.15.97:0/3510149403 wait complete. 2024-03-27T08:41:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.981+0000 7f186039d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.982+0000 7f186039d640 1 Processor -- start 2024-03-27T08:41:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.982+0000 7f186039d640 1 -- start start 2024-03-27T08:41:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.983+0000 7f186039d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 0x7f185811cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.983+0000 7f186039d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f185811e560 con 0x7f185806e4b0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f185e139640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 0x7f185811cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f185e139640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 0x7f185811cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60830/0 (socket says 172.21.15.97:60830) 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f185e139640 1 -- 172.21.15.97:0/633207013 learned_addr learned my addr 172.21.15.97:0/633207013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f185e139640 1 -- 172.21.15.97:0/633207013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185812ed20 con 0x7f185806e4b0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f185e139640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 0x7f185811cc50 crc :-1 s=READY pgs=13026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1854005a60 con 0x7f185806e4b0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1854005bc0 con 0x7f185806e4b0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185809b640 con 0x7f185806e4b0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1854005e60 con 0x7f185806e4b0 2024-03-27T08:41:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.984+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185809b2c0 con 0x7f185806e4b0 2024-03-27T08:41:39.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f185811fda0 con 0x7f185806e4b0 2024-03-27T08:41:39.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f18540070b0 con 0x7f185806e4b0 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f184effd640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f183c0387a0 0x7f183c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f1854048220 con 0x7f185806e4b0 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f184effd640 1 --2- 172.21.15.97:0/633207013 >> v2:172.21.15.190:6809/239686547 conn(0x7f183c03e860 0x7f183c040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 --> v2:172.21.15.190:6809/239686547 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f183c0413e0 con 0x7f183c03e860 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.985+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=714) ==== 24+0+0 (crc 0 0 0) 0x7f185404a710 con 0x7f185806e4b0 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.986+0000 7f185d938640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f183c0387a0 0x7f183c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.986+0000 7f185d938640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f183c0387a0 0x7f183c03acb0 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.987+0000 7f185e93a640 1 --2- 172.21.15.97:0/633207013 >> v2:172.21.15.190:6809/239686547 conn(0x7f183c03e860 0x7f183c040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:39.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.991+0000 7f185e93a640 1 --2- 172.21.15.97:0/633207013 >> v2:172.21.15.190:6809/239686547 conn(0x7f183c03e860 0x7f183c040d50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:39.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.992+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== osd.2 v2:172.21.15.190:6809/239686547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f183c0413e0 con 0x7f183c03e860 2024-03-27T08:41:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.996+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 --> v2:172.21.15.190:6809/239686547 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f18580a16c0 con 0x7f183c03e860 2024-03-27T08:41:39.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f184effd640 1 -- 172.21.15.97:0/633207013 <== osd.2 v2:172.21.15.190:6809/239686547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f18580a16c0 con 0x7f183c03e860 2024-03-27T08:41:39.990 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T08:41:39.990 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "" 2024-03-27T08:41:39.990 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 >> v2:172.21.15.190:6809/239686547 conn(0x7f183c03e860 msgr2=0x7f183c040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 --2- 172.21.15.97:0/633207013 >> v2:172.21.15.190:6809/239686547 conn(0x7f183c03e860 0x7f183c040d50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f183c0387a0 msgr2=0x7f183c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f183c0387a0 0x7f183c03acb0 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 msgr2=0x7f185811cc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 0x7f185811cc50 crc :-1 s=READY pgs=13026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 shutdown_connections 2024-03-27T08:41:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 --2- 172.21.15.97:0/633207013 >> v2:172.21.15.190:6809/239686547 conn(0x7f183c03e860 0x7f183c040d50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f183c0387a0 0x7f183c03acb0 unknown :-1 s=CLOSED pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 --2- 172.21.15.97:0/633207013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185806e4b0 0x7f185811cc50 unknown :-1 s=CLOSED pgs=13026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 >> 172.21.15.97:0/633207013 conn(0x7f185809d1d0 msgr2=0x7f1858132690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 shutdown_connections 2024-03-27T08:41:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:39.998+0000 7f186039d640 1 -- 172.21.15.97:0/633207013 wait complete. 2024-03-27T08:41:40.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-27T08:41:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.116+0000 7f863131d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.122+0000 7f863131d640 1 Processor -- start 2024-03-27T08:41:40.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.122+0000 7f863131d640 1 -- start start 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f863131d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f862c12f5c0 0x7f862c12f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f863131d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862c0732c0 con 0x7f862c132520 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f863131d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f862c09eeb0 con 0x7f862c12f5c0 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f862affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f862c12f5c0 0x7f862c12f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f8623fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f862c132520 0x7f862c132930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38984/0 (socket says 172.21.15.97:38984) 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f862affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f862c12f5c0 0x7f862c12f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60844/0 (socket says 172.21.15.97:60844) 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.123+0000 7f8623fff640 1 -- 172.21.15.97:0/1364098997 learned_addr learned my addr 172.21.15.97:0/1364098997 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.124+0000 7f862affd640 1 -- 172.21.15.97:0/1364098997 >> v1:172.21.15.97:6789/0 conn(0x7f862c132520 legacy=0x7f862c132930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.124+0000 7f862affd640 1 -- 172.21.15.97:0/1364098997 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c0a4820 con 0x7f862c12f5c0 2024-03-27T08:41:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.124+0000 7f862affd640 1 --2- 172.21.15.97:0/1364098997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c12f9d0 crc :-1 s=READY pgs=13028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf419d98dc9faa99 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.124+0000 7f862a7fc640 1 -- 172.21.15.97:0/1364098997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f861c009d90 con 0x7f862c12f5c0 2024-03-27T08:41:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.124+0000 7f862a7fc640 1 -- 172.21.15.97:0/1364098997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f861c00e230 con 0x7f862c12f5c0 2024-03-27T08:41:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.124+0000 7f862a7fc640 1 -- 172.21.15.97:0/1364098997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f861c00e4d0 con 0x7f862c12f5c0 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 -- 172.21.15.97:0/1364098997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 msgr2=0x7f862c12f9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 --2- 172.21.15.97:0/1364098997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c12f9d0 crc :-1 s=READY pgs=13028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 -- 172.21.15.97:0/1364098997 shutdown_connections 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 --2- 172.21.15.97:0/1364098997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c12f9d0 unknown :-1 s=CLOSED pgs=13028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 -- 172.21.15.97:0/1364098997 >> 172.21.15.97:0/1364098997 conn(0x7f862c0a29e0 msgr2=0x7f862c0a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 -- 172.21.15.97:0/1364098997 shutdown_connections 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 1 -- 172.21.15.97:0/1364098997 wait complete. 2024-03-27T08:41:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.125+0000 7f863131d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.126+0000 7f863131d640 1 Processor -- start 2024-03-27T08:41:40.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.126+0000 7f863131d640 1 -- start start 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.127+0000 7f863131d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c11dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.127+0000 7f863131d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f862c120a60 con 0x7f862c12f5c0 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.127+0000 7f862affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c11dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.127+0000 7f862affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c11dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60850/0 (socket says 172.21.15.97:60850) 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.127+0000 7f862affd640 1 -- 172.21.15.97:0/1193210968 learned_addr learned my addr 172.21.15.97:0/1193210968 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f862affd640 1 -- 172.21.15.97:0/1193210968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c120e90 con 0x7f862c12f5c0 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f862affd640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c11dfa0 crc :-1 s=READY pgs=13029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f861c014540 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f861c0146a0 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c11fc00 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f861c014940 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.128+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c11f1c0 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.129+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c121fe0 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.129+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f861c014af0 con 0x7f862c12f5c0 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.129+0000 7f86237fe640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f860c0387a0 0x7f860c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.129+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f861c019040 con 0x7f862c12f5c0 2024-03-27T08:41:40.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.129+0000 7f8623fff640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f860c0387a0 0x7f860c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.130+0000 7f8623fff640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f860c0387a0 0x7f860c03acb0 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.134+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f861c018190 con 0x7f862c12f5c0 2024-03-27T08:41:40.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.312+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f862c120bf0 con 0x7f860c0387a0 2024-03-27T08:41:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.312+0000 7f86237fe640 1 -- 172.21.15.97:0/1193210968 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f862c120bf0 con 0x7f860c0387a0 2024-03-27T08:41:40.304 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2024-03-27T08:41:40.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f860c0387a0 msgr2=0x7f860c03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f860c0387a0 0x7f860c03acb0 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 msgr2=0x7f862c11dfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c11dfa0 crc :-1 s=READY pgs=13029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 shutdown_connections 2024-03-27T08:41:40.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f860c0387a0 0x7f860c03acb0 unknown :-1 s=CLOSED pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 --2- 172.21.15.97:0/1193210968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c12f5c0 0x7f862c11dfa0 unknown :-1 s=CLOSED pgs=13029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.314+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 >> 172.21.15.97:0/1193210968 conn(0x7f862c0a29e0 msgr2=0x7f862c0a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.315+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 shutdown_connections 2024-03-27T08:41:40.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.315+0000 7f863131d640 1 -- 172.21.15.97:0/1193210968 wait complete. 2024-03-27T08:41:40.318 INFO:tasks.ceph:Scrubbing osd.3 2024-03-27T08:41:40.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-27T08:41:40.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.434+0000 7f3c5519d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.443+0000 7f3c5519d640 1 Processor -- start 2024-03-27T08:41:40.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.443+0000 7f3c5519d640 1 -- start start 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c5519d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c50133cc0 0x7f3c501360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c5519d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c5006f2c0 con 0x7f3c5006e4f0 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c5519d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3c50130950 con 0x7f3c50133cc0 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c47fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c50133cc0 0x7f3c501360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c4ed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3c5006e4f0 0x7f3c50130f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38990/0 (socket says 172.21.15.97:38990) 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c47fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c50133cc0 0x7f3c501360e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60858/0 (socket says 172.21.15.97:60858) 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c4ed9d640 1 -- 172.21.15.97:0/22902048 learned_addr learned my addr 172.21.15.97:0/22902048 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c47fff640 1 -- 172.21.15.97:0/22902048 >> v1:172.21.15.97:6789/0 conn(0x7f3c5006e4f0 legacy=0x7f3c50130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.444+0000 7f3c47fff640 1 -- 172.21.15.97:0/22902048 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c50130c60 con 0x7f3c50133cc0 2024-03-27T08:41:40.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.445+0000 7f3c47fff640 1 --2- 172.21.15.97:0/22902048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50133cc0 0x7f3c501360e0 crc :-1 s=READY pgs=13031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bab2e48582b3af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.445+0000 7f3c4e59c640 1 -- 172.21.15.97:0/22902048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3c3800ad20 con 0x7f3c50133cc0 2024-03-27T08:41:40.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.445+0000 7f3c4e59c640 1 -- 172.21.15.97:0/22902048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3c3800ae80 con 0x7f3c50133cc0 2024-03-27T08:41:40.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.445+0000 7f3c4e59c640 1 -- 172.21.15.97:0/22902048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3c3800f470 con 0x7f3c50133cc0 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.445+0000 7f3c5519d640 1 -- 172.21.15.97:0/22902048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50133cc0 msgr2=0x7f3c501360e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.445+0000 7f3c5519d640 1 --2- 172.21.15.97:0/22902048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50133cc0 0x7f3c501360e0 crc :-1 s=READY pgs=13031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.446+0000 7f3c5519d640 1 -- 172.21.15.97:0/22902048 shutdown_connections 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.446+0000 7f3c5519d640 1 --2- 172.21.15.97:0/22902048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50133cc0 0x7f3c501360e0 unknown :-1 s=CLOSED pgs=13031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.446+0000 7f3c5519d640 1 -- 172.21.15.97:0/22902048 >> 172.21.15.97:0/22902048 conn(0x7f3c5009d1d0 msgr2=0x7f3c5009d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.446+0000 7f3c5519d640 1 -- 172.21.15.97:0/22902048 shutdown_connections 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.446+0000 7f3c5519d640 1 -- 172.21.15.97:0/22902048 wait complete. 2024-03-27T08:41:40.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.446+0000 7f3c5519d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.447+0000 7f3c5519d640 1 Processor -- start 2024-03-27T08:41:40.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.447+0000 7f3c5519d640 1 -- start start 2024-03-27T08:41:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.448+0000 7f3c5519d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 0x7f3c50072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.448+0000 7f3c5519d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5011d600 con 0x7f3c5006e4f0 2024-03-27T08:41:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.448+0000 7f3c4ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 0x7f3c50072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.448+0000 7f3c4ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 0x7f3c50072790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60862/0 (socket says 172.21.15.97:60862) 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.448+0000 7f3c4ed9d640 1 -- 172.21.15.97:0/3344744914 learned_addr learned my addr 172.21.15.97:0/3344744914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c4ed9d640 1 -- 172.21.15.97:0/3344744914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c500721d0 con 0x7f3c5006e4f0 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c4ed9d640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 0x7f3c50072790 crc :-1 s=READY pgs=13032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3c40005a60 con 0x7f3c5006e4f0 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3c40005bc0 con 0x7f3c5006e4f0 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5011ba60 con 0x7f3c5006e4f0 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3c40005e60 con 0x7f3c5006e4f0 2024-03-27T08:41:40.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.449+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5011c9f0 con 0x7f3c5006e4f0 2024-03-27T08:41:40.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c50072060 con 0x7f3c5006e4f0 2024-03-27T08:41:40.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f3c400070b0 con 0x7f3c5006e4f0 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c477fe640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3c300387a0 0x7f3c3003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f3c40048220 con 0x7f3c5006e4f0 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c477fe640 1 --2- 172.21.15.97:0/3344744914 >> v2:172.21.15.190:6805/971408905 conn(0x7f3c3003e860 0x7f3c30040d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 --> v2:172.21.15.190:6805/971408905 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c300413e0 con 0x7f3c3003e860 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.450+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=714) ==== 24+0+0 (crc 0 0 0) 0x7f3c4004a710 con 0x7f3c5006e4f0 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.451+0000 7f3c47fff640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3c300387a0 0x7f3c3003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.451+0000 7f3c47fff640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3c300387a0 0x7f3c3003acb0 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.451+0000 7f3c4f59e640 1 --2- 172.21.15.97:0/3344744914 >> v2:172.21.15.190:6805/971408905 conn(0x7f3c3003e860 0x7f3c30040d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-27T08:41:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.454+0000 7f3c4f59e640 1 --2- 172.21.15.97:0/3344744914 >> v2:172.21.15.190:6805/971408905 conn(0x7f3c3003e860 0x7f3c30040d50 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.456+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== osd.3 v2:172.21.15.190:6805/971408905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f3c300413e0 con 0x7f3c3003e860 2024-03-27T08:41:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.459+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 --> v2:172.21.15.190:6805/971408905 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c5009cd50 con 0x7f3c3003e860 2024-03-27T08:41:40.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c477fe640 1 -- 172.21.15.97:0/3344744914 <== osd.3 v2:172.21.15.190:6805/971408905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f3c4800a820 con 0x7f3c3003e860 2024-03-27T08:41:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T08:41:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "" 2024-03-27T08:41:40.453 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 >> v2:172.21.15.190:6805/971408905 conn(0x7f3c3003e860 msgr2=0x7f3c30040d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 --2- 172.21.15.97:0/3344744914 >> v2:172.21.15.190:6805/971408905 conn(0x7f3c3003e860 0x7f3c30040d50 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3c300387a0 msgr2=0x7f3c3003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3c300387a0 0x7f3c3003acb0 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 msgr2=0x7f3c50072790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 0x7f3c50072790 crc :-1 s=READY pgs=13032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 shutdown_connections 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 --2- 172.21.15.97:0/3344744914 >> v2:172.21.15.190:6805/971408905 conn(0x7f3c3003e860 0x7f3c30040d50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f3c300387a0 0x7f3c3003acb0 unknown :-1 s=CLOSED pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 --2- 172.21.15.97:0/3344744914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5006e4f0 0x7f3c50072790 unknown :-1 s=CLOSED pgs=13032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 >> 172.21.15.97:0/3344744914 conn(0x7f3c5009d1d0 msgr2=0x7f3c5012f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.461+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 shutdown_connections 2024-03-27T08:41:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.462+0000 7f3c5519d640 1 -- 172.21.15.97:0/3344744914 wait complete. 2024-03-27T08:41:40.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-27T08:41:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.579+0000 7f8ad3e24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.584+0000 7f8ad3e24640 1 Processor -- start 2024-03-27T08:41:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.584+0000 7f8ad3e24640 1 -- start start 2024-03-27T08:41:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.585+0000 7f8ad3e24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8acc12da20 0x7f8acc09c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.585+0000 7f8ad3e24640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc12e2b0 con 0x7f8acc09ca40 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.585+0000 7f8ad3e24640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc06e4d0 con 0x7f8acc12da20 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad1bc0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8acc12da20 0x7f8acc09c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad1bc0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8acc12da20 0x7f8acc09c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60876/0 (socket says 172.21.15.97:60876) 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad13bf640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8acc09ca40 0x7f8acc1309a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38994/0 (socket says 172.21.15.97:38994) 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad1bc0640 1 -- 172.21.15.97:0/1425644779 learned_addr learned my addr 172.21.15.97:0/1425644779 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad1bc0640 1 -- 172.21.15.97:0/1425644779 >> v1:172.21.15.97:6789/0 conn(0x7f8acc09ca40 legacy=0x7f8acc1309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad1bc0640 1 -- 172.21.15.97:0/1425644779 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc12dfa0 con 0x7f8acc12da20 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad1bc0640 1 --2- 172.21.15.97:0/1425644779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc12da20 0x7f8acc09c500 crc :-1 s=READY pgs=13034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0318a96c842e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad0bbe640 1 -- 172.21.15.97:0/1425644779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8ac400f040 con 0x7f8acc12da20 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad0bbe640 1 -- 172.21.15.97:0/1425644779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8ac400a1e0 con 0x7f8acc12da20 2024-03-27T08:41:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.586+0000 7f8ad0bbe640 1 -- 172.21.15.97:0/1425644779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8ac4008710 con 0x7f8acc12da20 2024-03-27T08:41:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 -- 172.21.15.97:0/1425644779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc12da20 msgr2=0x7f8acc09c500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 --2- 172.21.15.97:0/1425644779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc12da20 0x7f8acc09c500 crc :-1 s=READY pgs=13034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 -- 172.21.15.97:0/1425644779 shutdown_connections 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 --2- 172.21.15.97:0/1425644779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc12da20 0x7f8acc09c500 unknown :-1 s=CLOSED pgs=13034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 -- 172.21.15.97:0/1425644779 >> 172.21.15.97:0/1425644779 conn(0x7f8acc09d1d0 msgr2=0x7f8acc09d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 -- 172.21.15.97:0/1425644779 shutdown_connections 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 1 -- 172.21.15.97:0/1425644779 wait complete. 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.587+0000 7f8ad3e24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.589+0000 7f8ad3e24640 1 Processor -- start 2024-03-27T08:41:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.589+0000 7f8ad3e24640 1 -- start start 2024-03-27T08:41:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad3e24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 0x7f8acc119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad3e24640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc06a9e0 con 0x7f8acc09ca40 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad1bc0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 0x7f8acc119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad1bc0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 0x7f8acc119fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60886/0 (socket says 172.21.15.97:60886) 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad1bc0640 1 -- 172.21.15.97:0/2104901619 learned_addr learned my addr 172.21.15.97:0/2104901619 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad1bc0640 1 -- 172.21.15.97:0/2104901619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc119980 con 0x7f8acc09ca40 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad1bc0640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 0x7f8acc119fd0 crc :-1 s=READY pgs=13035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8ac400f040 con 0x7f8acc09ca40 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8ac400a5e0 con 0x7f8acc09ca40 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc11d540 con 0x7f8acc09ca40 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8ac4012980 con 0x7f8acc09ca40 2024-03-27T08:41:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.590+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc11c700 con 0x7f8acc09ca40 2024-03-27T08:41:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.591+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc11c8f0 con 0x7f8acc09ca40 2024-03-27T08:41:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.591+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f8ac4012b30 con 0x7f8acc09ca40 2024-03-27T08:41:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.592+0000 7f8ac27fc640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8ab00387a0 0x7f8ab003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.592+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f8ac404aa30 con 0x7f8acc09ca40 2024-03-27T08:41:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.592+0000 7f8ad13bf640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8ab00387a0 0x7f8ab003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.592+0000 7f8ad13bf640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8ab00387a0 0x7f8ab003acb0 crc :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.597+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f8ac40153e0 con 0x7f8acc09ca40 2024-03-27T08:41:40.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.773+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f8acc09ce50 con 0x7f8ab00387a0 2024-03-27T08:41:40.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.773+0000 7f8ac27fc640 1 -- 172.21.15.97:0/2104901619 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f8abc005870 con 0x7f8ab00387a0 2024-03-27T08:41:40.765 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8ab00387a0 msgr2=0x7f8ab003acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8ab00387a0 0x7f8ab003acb0 crc :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 msgr2=0x7f8acc119fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 0x7f8acc119fd0 crc :-1 s=READY pgs=13035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 shutdown_connections 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f8ab00387a0 0x7f8ab003acb0 unknown :-1 s=CLOSED pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 --2- 172.21.15.97:0/2104901619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8acc09ca40 0x7f8acc119fd0 unknown :-1 s=CLOSED pgs=13035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 >> 172.21.15.97:0/2104901619 conn(0x7f8acc09d1d0 msgr2=0x7f8acc09aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 shutdown_connections 2024-03-27T08:41:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.775+0000 7f8ad3e24640 1 -- 172.21.15.97:0/2104901619 wait complete. 2024-03-27T08:41:40.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:41:40.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.894+0000 7f39c8e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.898+0000 7f39c8e6d640 1 Processor -- start 2024-03-27T08:41:40.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.898+0000 7f39c8e6d640 1 -- start start 2024-03-27T08:41:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.899+0000 7f39c8e6d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39c412f360 0x7f39c412f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.899+0000 7f39c8e6d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39c409edb0 con 0x7f39c4132330 2024-03-27T08:41:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.899+0000 7f39c8e6d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f39c409af10 con 0x7f39c412f360 2024-03-27T08:41:40.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39c412f360 0x7f39c412f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39c412f360 0x7f39c412f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60896/0 (socket says 172.21.15.97:60896) 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c1d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f39c4132330 0x7f39c4132740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38998/0 (socket says 172.21.15.97:38998) 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c259c640 1 -- 172.21.15.97:0/3622401087 learned_addr learned my addr 172.21.15.97:0/3622401087 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c259c640 1 -- 172.21.15.97:0/3622401087 >> v1:172.21.15.97:6789/0 conn(0x7f39c4132330 legacy=0x7f39c4132740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c259c640 1 -- 172.21.15.97:0/3622401087 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c4073520 con 0x7f39c412f360 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c159a640 1 -- 172.21.15.97:0/3622401087 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415814043 0 0) 0x7f39c409edb0 con 0x7f39c4132330 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c259c640 1 --2- 172.21.15.97:0/3622401087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c412f770 crc :-1 s=READY pgs=13037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e41e0bf153569 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c159a640 1 -- 172.21.15.97:0/3622401087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f39b400e040 con 0x7f39c412f360 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c159a640 1 -- 172.21.15.97:0/3622401087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f39b40091e0 con 0x7f39c412f360 2024-03-27T08:41:40.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.900+0000 7f39c159a640 1 -- 172.21.15.97:0/3622401087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f39b4007730 con 0x7f39c412f360 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 -- 172.21.15.97:0/3622401087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 msgr2=0x7f39c412f770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 --2- 172.21.15.97:0/3622401087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c412f770 crc :-1 s=READY pgs=13037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 -- 172.21.15.97:0/3622401087 shutdown_connections 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 --2- 172.21.15.97:0/3622401087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c412f770 unknown :-1 s=CLOSED pgs=13037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 -- 172.21.15.97:0/3622401087 >> 172.21.15.97:0/3622401087 conn(0x7f39c409d1d0 msgr2=0x7f39c409d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 -- 172.21.15.97:0/3622401087 shutdown_connections 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 1 -- 172.21.15.97:0/3622401087 wait complete. 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.901+0000 7f39c8e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:41:40.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.903+0000 7f39c8e6d640 1 Processor -- start 2024-03-27T08:41:40.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.903+0000 7f39c8e6d640 1 -- start start 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c8e6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c411aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c8e6d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39c412edc0 con 0x7f39c412f360 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c411aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c411aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60904/0 (socket says 172.21.15.97:60904) 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c259c640 1 -- 172.21.15.97:0/419742048 learned_addr learned my addr 172.21.15.97:0/419742048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c259c640 1 -- 172.21.15.97:0/419742048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c412ead0 con 0x7f39c412f360 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c259c640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c411aa70 crc :-1 s=READY pgs=13038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f39b400e040 con 0x7f39c412f360 2024-03-27T08:41:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f39b40094d0 con 0x7f39c412f360 2024-03-27T08:41:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c411eb30 con 0x7f39c412f360 2024-03-27T08:41:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f39b4011920 con 0x7f39c412f360 2024-03-27T08:41:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.904+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c406a9e0 con 0x7f39c412f360 2024-03-27T08:41:40.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.905+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f39b4011ad0 con 0x7f39c412f360 2024-03-27T08:41:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.905+0000 7f39b2ffd640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f39a40387a0 0x7f39a403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:41:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.905+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f39b40497d0 con 0x7f39c412f360 2024-03-27T08:41:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.906+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c411cf80 con 0x7f39c412f360 2024-03-27T08:41:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.906+0000 7f39c1d9b640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f39a40387a0 0x7f39a403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:41:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.906+0000 7f39c1d9b640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f39a40387a0 0x7f39a403acb0 crc :-1 s=READY pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:41:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:40.911+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f39b40143e0 con 0x7f39c412f360 2024-03-27T08:41:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.083+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39c4072e50 con 0x7f39a40387a0 2024-03-27T08:41:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.086+0000 7f39b2ffd640 1 -- 172.21.15.97:0/419742048 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107144 (crc 0 0 0) 0x7f39c4072e50 con 0x7f39a40387a0 2024-03-27T08:41:41.079 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:41:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:41:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f39a40387a0 msgr2=0x7f39a403acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f39a40387a0 0x7f39a403acb0 crc :-1 s=READY pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 msgr2=0x7f39c411aa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c411aa70 crc :-1 s=READY pgs=13038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 shutdown_connections 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f39a40387a0 0x7f39a403acb0 unknown :-1 s=CLOSED pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 --2- 172.21.15.97:0/419742048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c412f360 0x7f39c411aa70 unknown :-1 s=CLOSED pgs=13038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.088+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 >> 172.21.15.97:0/419742048 conn(0x7f39c409d1d0 msgr2=0x7f39c4130130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.089+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 shutdown_connections 2024-03-27T08:41:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:41:41.089+0000 7f39c8e6d640 1 -- 172.21.15.97:0/419742048 wait complete. 2024-03-27T08:41:41.092 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":1265,"stamp":"2024-03-27T08:41:40.186008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":312864,"kb_used_data":14912,"kb_used_omap":0,"kb_used_meta":297920,"kb_avail":419117536,"statfs":{"total":429496729600,"available":429176356864,"internally_reserved":0,"allocated":15269888,"data_stored":6607826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":305070080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001850"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1276,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.930862+0000","last_change":"2024-03-27T08:27:58.956969+0000","last_active":"2024-03-27T08:41:35.930862+0000","last_peered":"2024-03-27T08:41:35.930862+0000","last_clean":"2024-03-27T08:41:35.930862+0000","last_became_active":"2024-03-27T08:27:58.944924+0000","last_became_peered":"2024-03-27T08:27:58.944924+0000","last_unstale":"2024-03-27T08:41:35.930862+0000","last_undegraded":"2024-03-27T08:41:35.930862+0000","last_fullsized":"2024-03-27T08:41:35.930862+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.363680+0000","last_change":"2024-03-27T08:27:59.042742+0000","last_active":"2024-03-27T08:41:35.363680+0000","last_peered":"2024-03-27T08:41:35.363680+0000","last_clean":"2024-03-27T08:41:35.363680+0000","last_became_active":"2024-03-27T08:27:59.029389+0000","last_became_peered":"2024-03-27T08:27:59.029389+0000","last_unstale":"2024-03-27T08:41:35.363680+0000","last_undegraded":"2024-03-27T08:41:35.363680+0000","last_fullsized":"2024-03-27T08:41:35.363680+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1259,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.939786+0000","last_change":"2024-03-27T08:27:58.946023+0000","last_active":"2024-03-27T08:41:34.939786+0000","last_peered":"2024-03-27T08:41:34.939786+0000","last_clean":"2024-03-27T08:41:34.939786+0000","last_became_active":"2024-03-27T08:27:58.934198+0000","last_became_peered":"2024-03-27T08:27:58.934198+0000","last_unstale":"2024-03-27T08:41:34.939786+0000","last_undegraded":"2024-03-27T08:41:34.939786+0000","last_fullsized":"2024-03-27T08:41:34.939786+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.302330+0000","last_change":"2024-03-27T08:27:58.982774+0000","last_active":"2024-03-27T08:41:35.302330+0000","last_peered":"2024-03-27T08:41:35.302330+0000","last_clean":"2024-03-27T08:41:35.302330+0000","last_became_active":"2024-03-27T08:27:58.963619+0000","last_became_peered":"2024-03-27T08:27:58.963619+0000","last_unstale":"2024-03-27T08:41:35.302330+0000","last_undegraded":"2024-03-27T08:41:35.302330+0000","last_fullsized":"2024-03-27T08:41:35.302330+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1276,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.940100+0000","last_change":"2024-03-27T08:27:58.974218+0000","last_active":"2024-03-27T08:41:35.940100+0000","last_peered":"2024-03-27T08:41:35.940100+0000","last_clean":"2024-03-27T08:41:35.940100+0000","last_became_active":"2024-03-27T08:27:58.967369+0000","last_became_peered":"2024-03-27T08:27:58.967369+0000","last_unstale":"2024-03-27T08:41:35.940100+0000","last_undegraded":"2024-03-27T08:41:35.940100+0000","last_fullsized":"2024-03-27T08:41:35.940100+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.426985+0000","last_change":"2024-03-27T08:27:58.970092+0000","last_active":"2024-03-27T08:41:35.426985+0000","last_peered":"2024-03-27T08:41:35.426985+0000","last_clean":"2024-03-27T08:41:35.426985+0000","last_became_active":"2024-03-27T08:27:58.956201+0000","last_became_peered":"2024-03-27T08:27:58.956201+0000","last_unstale":"2024-03-27T08:41:35.426985+0000","last_undegraded":"2024-03-27T08:41:35.426985+0000","last_fullsized":"2024-03-27T08:41:35.426985+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.392663+0000","last_change":"2024-03-27T08:27:59.003207+0000","last_active":"2024-03-27T08:41:35.392663+0000","last_peered":"2024-03-27T08:41:35.392663+0000","last_clean":"2024-03-27T08:41:35.392663+0000","last_became_active":"2024-03-27T08:27:58.981709+0000","last_became_peered":"2024-03-27T08:27:58.981709+0000","last_unstale":"2024-03-27T08:41:35.392663+0000","last_undegraded":"2024-03-27T08:41:35.392663+0000","last_fullsized":"2024-03-27T08:41:35.392663+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.412806+0000","last_change":"2024-03-27T08:27:58.997466+0000","last_active":"2024-03-27T08:41:35.412806+0000","last_peered":"2024-03-27T08:41:35.412806+0000","last_clean":"2024-03-27T08:41:35.412806+0000","last_became_active":"2024-03-27T08:27:58.977694+0000","last_became_peered":"2024-03-27T08:27:58.977694+0000","last_unstale":"2024-03-27T08:41:35.412806+0000","last_undegraded":"2024-03-27T08:41:35.412806+0000","last_fullsized":"2024-03-27T08:41:35.412806+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.331363+0000","last_change":"2024-03-27T08:27:59.001487+0000","last_active":"2024-03-27T08:41:35.331363+0000","last_peered":"2024-03-27T08:41:35.331363+0000","last_clean":"2024-03-27T08:41:35.331363+0000","last_became_active":"2024-03-27T08:27:58.979333+0000","last_became_peered":"2024-03-27T08:27:58.979333+0000","last_unstale":"2024-03-27T08:41:35.331363+0000","last_undegraded":"2024-03-27T08:41:35.331363+0000","last_fullsized":"2024-03-27T08:41:35.331363+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1259,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.948156+0000","last_change":"2024-03-27T08:27:58.954275+0000","last_active":"2024-03-27T08:41:34.948156+0000","last_peered":"2024-03-27T08:41:34.948156+0000","last_clean":"2024-03-27T08:41:34.948156+0000","last_became_active":"2024-03-27T08:27:58.939180+0000","last_became_peered":"2024-03-27T08:27:58.939180+0000","last_unstale":"2024-03-27T08:41:34.948156+0000","last_undegraded":"2024-03-27T08:41:34.948156+0000","last_fullsized":"2024-03-27T08:41:34.948156+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.301125+0000","last_change":"2024-03-27T08:27:58.988220+0000","last_active":"2024-03-27T08:41:35.301125+0000","last_peered":"2024-03-27T08:41:35.301125+0000","last_clean":"2024-03-27T08:41:35.301125+0000","last_became_active":"2024-03-27T08:27:58.955512+0000","last_became_peered":"2024-03-27T08:27:58.955512+0000","last_unstale":"2024-03-27T08:41:35.301125+0000","last_undegraded":"2024-03-27T08:41:35.301125+0000","last_fullsized":"2024-03-27T08:41:35.301125+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.388016+0000","last_change":"2024-03-27T08:27:59.103582+0000","last_active":"2024-03-27T08:41:35.388016+0000","last_peered":"2024-03-27T08:41:35.388016+0000","last_clean":"2024-03-27T08:41:35.388016+0000","last_became_active":"2024-03-27T08:27:59.098593+0000","last_became_peered":"2024-03-27T08:27:59.098593+0000","last_unstale":"2024-03-27T08:41:35.388016+0000","last_undegraded":"2024-03-27T08:41:35.388016+0000","last_fullsized":"2024-03-27T08:41:35.388016+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.355003+0000","last_change":"2024-03-27T08:27:58.999586+0000","last_active":"2024-03-27T08:41:35.355003+0000","last_peered":"2024-03-27T08:41:35.355003+0000","last_clean":"2024-03-27T08:41:35.355003+0000","last_became_active":"2024-03-27T08:27:58.973352+0000","last_became_peered":"2024-03-27T08:27:58.973352+0000","last_unstale":"2024-03-27T08:41:35.355003+0000","last_undegraded":"2024-03-27T08:41:35.355003+0000","last_fullsized":"2024-03-27T08:41:35.355003+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.352873+0000","last_change":"2024-03-27T08:27:59.070788+0000","last_active":"2024-03-27T08:41:35.352873+0000","last_peered":"2024-03-27T08:41:35.352873+0000","last_clean":"2024-03-27T08:41:35.352873+0000","last_became_active":"2024-03-27T08:27:59.065991+0000","last_became_peered":"2024-03-27T08:27:59.065991+0000","last_unstale":"2024-03-27T08:41:35.352873+0000","last_undegraded":"2024-03-27T08:41:35.352873+0000","last_fullsized":"2024-03-27T08:41:35.352873+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.369300+0000","last_change":"2024-03-27T08:27:59.032231+0000","last_active":"2024-03-27T08:41:35.369300+0000","last_peered":"2024-03-27T08:41:35.369300+0000","last_clean":"2024-03-27T08:41:35.369300+0000","last_became_active":"2024-03-27T08:27:59.007454+0000","last_became_peered":"2024-03-27T08:27:59.007454+0000","last_unstale":"2024-03-27T08:41:35.369300+0000","last_undegraded":"2024-03-27T08:41:35.369300+0000","last_fullsized":"2024-03-27T08:41:35.369300+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.428433+0000","last_change":"2024-03-27T08:27:59.036310+0000","last_active":"2024-03-27T08:41:35.428433+0000","last_peered":"2024-03-27T08:41:35.428433+0000","last_clean":"2024-03-27T08:41:35.428433+0000","last_became_active":"2024-03-27T08:27:59.015211+0000","last_became_peered":"2024-03-27T08:27:59.015211+0000","last_unstale":"2024-03-27T08:41:35.428433+0000","last_undegraded":"2024-03-27T08:41:35.428433+0000","last_fullsized":"2024-03-27T08:41:35.428433+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.447077+0000","last_change":"2024-03-27T08:27:58.973584+0000","last_active":"2024-03-27T08:41:35.447077+0000","last_peered":"2024-03-27T08:41:35.447077+0000","last_clean":"2024-03-27T08:41:35.447077+0000","last_became_active":"2024-03-27T08:27:58.960673+0000","last_became_peered":"2024-03-27T08:27:58.960673+0000","last_unstale":"2024-03-27T08:41:35.447077+0000","last_undegraded":"2024-03-27T08:41:35.447077+0000","last_fullsized":"2024-03-27T08:41:35.447077+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.320778+0000","last_change":"2024-03-27T08:27:59.040676+0000","last_active":"2024-03-27T08:41:35.320778+0000","last_peered":"2024-03-27T08:41:35.320778+0000","last_clean":"2024-03-27T08:41:35.320778+0000","last_became_active":"2024-03-27T08:27:59.034744+0000","last_became_peered":"2024-03-27T08:27:59.034744+0000","last_unstale":"2024-03-27T08:41:35.320778+0000","last_undegraded":"2024-03-27T08:41:35.320778+0000","last_fullsized":"2024-03-27T08:41:35.320778+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1259,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.936076+0000","last_change":"2024-03-27T08:27:58.993514+0000","last_active":"2024-03-27T08:41:34.936076+0000","last_peered":"2024-03-27T08:41:34.936076+0000","last_clean":"2024-03-27T08:41:34.936076+0000","last_became_active":"2024-03-27T08:27:58.988938+0000","last_became_peered":"2024-03-27T08:27:58.988938+0000","last_unstale":"2024-03-27T08:41:34.936076+0000","last_undegraded":"2024-03-27T08:41:34.936076+0000","last_fullsized":"2024-03-27T08:41:34.936076+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1276,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.941271+0000","last_change":"2024-03-27T08:27:58.953456+0000","last_active":"2024-03-27T08:41:35.941271+0000","last_peered":"2024-03-27T08:41:35.941271+0000","last_clean":"2024-03-27T08:41:35.941271+0000","last_became_active":"2024-03-27T08:27:58.940428+0000","last_became_peered":"2024-03-27T08:27:58.940428+0000","last_unstale":"2024-03-27T08:41:35.941271+0000","last_undegraded":"2024-03-27T08:41:35.941271+0000","last_fullsized":"2024-03-27T08:41:35.941271+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1276,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.962538+0000","last_change":"2024-03-27T08:27:58.991121+0000","last_active":"2024-03-27T08:41:35.962538+0000","last_peered":"2024-03-27T08:41:35.962538+0000","last_clean":"2024-03-27T08:41:35.962538+0000","last_became_active":"2024-03-27T08:27:58.982971+0000","last_became_peered":"2024-03-27T08:27:58.982971+0000","last_unstale":"2024-03-27T08:41:35.962538+0000","last_undegraded":"2024-03-27T08:41:35.962538+0000","last_fullsized":"2024-03-27T08:41:35.962538+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1276,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.920186+0000","last_change":"2024-03-27T08:27:59.023638+0000","last_active":"2024-03-27T08:41:35.920186+0000","last_peered":"2024-03-27T08:41:35.920186+0000","last_clean":"2024-03-27T08:41:35.920186+0000","last_became_active":"2024-03-27T08:27:59.017861+0000","last_became_peered":"2024-03-27T08:27:59.017861+0000","last_unstale":"2024-03-27T08:41:35.920186+0000","last_undegraded":"2024-03-27T08:41:35.920186+0000","last_fullsized":"2024-03-27T08:41:35.920186+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1259,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.896337+0000","last_change":"2024-03-27T08:27:58.950095+0000","last_active":"2024-03-27T08:41:34.896337+0000","last_peered":"2024-03-27T08:41:34.896337+0000","last_clean":"2024-03-27T08:41:34.896337+0000","last_became_active":"2024-03-27T08:27:58.942454+0000","last_became_peered":"2024-03-27T08:27:58.942454+0000","last_unstale":"2024-03-27T08:41:34.896337+0000","last_undegraded":"2024-03-27T08:41:34.896337+0000","last_fullsized":"2024-03-27T08:41:34.896337+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.319972+0000","last_change":"2024-03-27T08:27:58.986152+0000","last_active":"2024-03-27T08:41:35.319972+0000","last_peered":"2024-03-27T08:41:35.319972+0000","last_clean":"2024-03-27T08:41:35.319972+0000","last_became_active":"2024-03-27T08:27:58.960185+0000","last_became_peered":"2024-03-27T08:27:58.960185+0000","last_unstale":"2024-03-27T08:41:35.319972+0000","last_undegraded":"2024-03-27T08:41:35.319972+0000","last_fullsized":"2024-03-27T08:41:35.319972+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.351782+0000","last_change":"2024-03-27T08:27:58.996597+0000","last_active":"2024-03-27T08:41:35.351782+0000","last_peered":"2024-03-27T08:41:35.351782+0000","last_clean":"2024-03-27T08:41:35.351782+0000","last_became_active":"2024-03-27T08:27:58.969756+0000","last_became_peered":"2024-03-27T08:27:58.969756+0000","last_unstale":"2024-03-27T08:41:35.351782+0000","last_undegraded":"2024-03-27T08:41:35.351782+0000","last_fullsized":"2024-03-27T08:41:35.351782+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.421001+0000","last_change":"2024-03-27T08:27:59.018353+0000","last_active":"2024-03-27T08:41:35.421001+0000","last_peered":"2024-03-27T08:41:35.421001+0000","last_clean":"2024-03-27T08:41:35.421001+0000","last_became_active":"2024-03-27T08:27:59.000404+0000","last_became_peered":"2024-03-27T08:27:59.000404+0000","last_unstale":"2024-03-27T08:41:35.421001+0000","last_undegraded":"2024-03-27T08:41:35.421001+0000","last_fullsized":"2024-03-27T08:41:35.421001+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1253,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.335216+0000","last_change":"2024-03-27T08:27:59.012490+0000","last_active":"2024-03-27T08:41:35.335216+0000","last_peered":"2024-03-27T08:41:35.335216+0000","last_clean":"2024-03-27T08:41:35.335216+0000","last_became_active":"2024-03-27T08:27:59.004002+0000","last_became_peered":"2024-03-27T08:27:59.004002+0000","last_unstale":"2024-03-27T08:41:35.335216+0000","last_undegraded":"2024-03-27T08:41:35.335216+0000","last_fullsized":"2024-03-27T08:41:35.335216+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.398548+0000","last_change":"2024-03-27T08:27:59.052618+0000","last_active":"2024-03-27T08:41:35.398548+0000","last_peered":"2024-03-27T08:41:35.398548+0000","last_clean":"2024-03-27T08:41:35.398548+0000","last_became_active":"2024-03-27T08:27:59.045303+0000","last_became_peered":"2024-03-27T08:27:59.045303+0000","last_unstale":"2024-03-27T08:41:35.398548+0000","last_undegraded":"2024-03-27T08:41:35.398548+0000","last_fullsized":"2024-03-27T08:41:35.398548+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.407275+0000","last_change":"2024-03-27T08:27:58.988986+0000","last_active":"2024-03-27T08:41:35.407275+0000","last_peered":"2024-03-27T08:41:35.407275+0000","last_clean":"2024-03-27T08:41:35.407275+0000","last_became_active":"2024-03-27T08:27:58.964911+0000","last_became_peered":"2024-03-27T08:27:58.964911+0000","last_unstale":"2024-03-27T08:41:35.407275+0000","last_undegraded":"2024-03-27T08:41:35.407275+0000","last_fullsized":"2024-03-27T08:41:35.407275+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.403996+0000","last_change":"2024-03-27T08:27:59.011613+0000","last_active":"2024-03-27T08:41:35.403996+0000","last_peered":"2024-03-27T08:41:35.403996+0000","last_clean":"2024-03-27T08:41:35.403996+0000","last_became_active":"2024-03-27T08:27:58.985998+0000","last_became_peered":"2024-03-27T08:27:58.985998+0000","last_unstale":"2024-03-27T08:41:35.403996+0000","last_undegraded":"2024-03-27T08:41:35.403996+0000","last_fullsized":"2024-03-27T08:41:35.403996+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1289,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.915797+0000","last_change":"2024-03-27T08:27:58.951044+0000","last_active":"2024-03-27T08:41:35.915797+0000","last_peered":"2024-03-27T08:41:35.915797+0000","last_clean":"2024-03-27T08:41:35.915797+0000","last_became_active":"2024-03-27T08:27:58.934959+0000","last_became_peered":"2024-03-27T08:27:58.934959+0000","last_unstale":"2024-03-27T08:41:35.915797+0000","last_undegraded":"2024-03-27T08:41:35.915797+0000","last_fullsized":"2024-03-27T08:41:35.915797+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1259,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.949691+0000","last_change":"2024-03-27T08:27:58.978772+0000","last_active":"2024-03-27T08:41:34.949691+0000","last_peered":"2024-03-27T08:41:34.949691+0000","last_clean":"2024-03-27T08:41:34.949691+0000","last_became_active":"2024-03-27T08:27:58.972299+0000","last_became_peered":"2024-03-27T08:27:58.972299+0000","last_unstale":"2024-03-27T08:41:34.949691+0000","last_undegraded":"2024-03-27T08:41:34.949691+0000","last_fullsized":"2024-03-27T08:41:34.949691+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:27:57.746239+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1865,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.924818+0000","last_change":"2024-03-27T08:23:09.066891+0000","last_active":"2024-03-27T08:41:35.924818+0000","last_peered":"2024-03-27T08:41:35.924818+0000","last_clean":"2024-03-27T08:41:35.924818+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:41:35.924818+0000","last_undegraded":"2024-03-27T08:41:35.924818+0000","last_fullsized":"2024-03-27T08:41:35.924818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1848,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.984120+0000","last_change":"2024-03-27T08:23:09.059885+0000","last_active":"2024-03-27T08:41:34.984120+0000","last_peered":"2024-03-27T08:41:34.984120+0000","last_clean":"2024-03-27T08:41:34.984120+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:41:34.984120+0000","last_undegraded":"2024-03-27T08:41:34.984120+0000","last_fullsized":"2024-03-27T08:41:34.984120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1854,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.439375+0000","last_change":"2024-03-27T08:23:09.052784+0000","last_active":"2024-03-27T08:41:35.439375+0000","last_peered":"2024-03-27T08:41:35.439375+0000","last_clean":"2024-03-27T08:41:35.439375+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:41:35.439375+0000","last_undegraded":"2024-03-27T08:41:35.439375+0000","last_fullsized":"2024-03-27T08:41:35.439375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1848,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:34.897216+0000","last_change":"2024-03-27T08:23:09.053884+0000","last_active":"2024-03-27T08:41:34.897216+0000","last_peered":"2024-03-27T08:41:34.897216+0000","last_clean":"2024-03-27T08:41:34.897216+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:41:34.897216+0000","last_undegraded":"2024-03-27T08:41:34.897216+0000","last_fullsized":"2024-03-27T08:41:34.897216+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1890,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.442527+0000","last_change":"2024-03-27T08:23:09.061623+0000","last_active":"2024-03-27T08:41:35.442527+0000","last_peered":"2024-03-27T08:41:35.442527+0000","last_clean":"2024-03-27T08:41:35.442527+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:41:35.442527+0000","last_undegraded":"2024-03-27T08:41:35.442527+0000","last_fullsized":"2024-03-27T08:41:35.442527+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1865,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.937359+0000","last_change":"2024-03-27T08:23:09.055461+0000","last_active":"2024-03-27T08:41:35.937359+0000","last_peered":"2024-03-27T08:41:35.937359+0000","last_clean":"2024-03-27T08:41:35.937359+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:41:35.937359+0000","last_undegraded":"2024-03-27T08:41:35.937359+0000","last_fullsized":"2024-03-27T08:41:35.937359+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1890,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.530427+0000","last_change":"2024-03-27T08:23:09.074614+0000","last_active":"2024-03-27T08:41:35.530427+0000","last_peered":"2024-03-27T08:41:35.530427+0000","last_clean":"2024-03-27T08:41:35.530427+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:41:35.530427+0000","last_undegraded":"2024-03-27T08:41:35.530427+0000","last_fullsized":"2024-03-27T08:41:35.530427+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1865,"reported_epoch":714,"state":"active+clean","last_fresh":"2024-03-27T08:41:36.027804+0000","last_change":"2024-03-27T08:23:09.065607+0000","last_active":"2024-03-27T08:41:36.027804+0000","last_peered":"2024-03-27T08:41:36.027804+0000","last_clean":"2024-03-27T08:41:36.027804+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:41:36.027804+0000","last_undegraded":"2024-03-27T08:41:36.027804+0000","last_fullsized":"2024-03-27T08:41:36.027804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:07.989354+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1987,"reported_epoch":713,"state":"active+clean","last_fresh":"2024-03-27T08:41:35.305933+0000","last_change":"2024-03-27T08:23:07.031764+0000","last_active":"2024-03-27T08:41:35.305933+0000","last_peered":"2024-03-27T08:41:35.305933+0000","last_clean":"2024-03-27T08:41:35.305933+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:41:35.305933+0000","last_undegraded":"2024-03-27T08:41:35.305933+0000","last_fullsized":"2024-03-27T08:41:35.305933+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:23:05.980792+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79544,"kb_used_data":4016,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778056,"statfs":{"total":107374182400,"available":107292729344,"internally_reserved":0,"allocated":4112384,"data_stored":1959909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83188,"kb_used_data":3500,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104774412,"statfs":{"total":107374182400,"available":107288997888,"internally_reserved":0,"allocated":3584000,"data_stored":1386442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66436,"kb_used_data":3388,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791164,"statfs":{"total":107374182400,"available":107306151936,"internally_reserved":0,"allocated":3469312,"data_stored":1313805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83696,"kb_used_data":4008,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104773904,"statfs":{"total":107374182400,"available":107288477696,"internally_reserved":0,"allocated":4104192,"data_stored":1947670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:41:41.096 INFO:tasks.ceph:pgid 75.1f last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.096 INFO:tasks.ceph:pgid 75.1e last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.1d last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.1c last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.1b last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.1a last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.19 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.18 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.17 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.16 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.15 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.097 INFO:tasks.ceph:pgid 75.14 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.13 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.12 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.11 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.10 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.f last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.e last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.d last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.c last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.098 INFO:tasks.ceph:pgid 75.b last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.a last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.9 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.8 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.7 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.6 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.5 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.4 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.3 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.099 INFO:tasks.ceph:pgid 75.2 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 75.1 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 75.0 last_scrub_stamp 2024-03-27T08:27:57.746239+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=27, tm_sec=57, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.100 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.101 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-27T08:23:07.989354+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=7, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.101 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-27T08:23:05.980792+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=23, tm_sec=5, tm_wday=2, tm_yday=87, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=27, tm_hour=8, tm_min=41, tm_sec=37, tm_wday=2, tm_yday=87, tm_isdst=0) 2024-03-27T08:41:41.101 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-27T08:41:44.881 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-27T08:41:44.888+0000 7f32301f9640 -1 log_channel(cluster) log [ERR] : Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-27T08:41:44.881 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-27T08:41:44.888+0000 7f32301f9640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 41 pgs inconsistent (PG_DAMAGED) 2024-03-27T08:42:01.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T08:42:01.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.228+0000 7f251f79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:01.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.233+0000 7f251f79d640 1 Processor -- start 2024-03-27T08:42:01.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.233+0000 7f251f79d640 1 -- start start 2024-03-27T08:42:01.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251f79d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f25181378c0 0x7f2518139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:01.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251f79d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25180a06a0 con 0x7f2518131650 2024-03-27T08:42:01.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251f79d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2518132170 con 0x7f25181378c0 2024-03-27T08:42:01.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251d539640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2518131650 0x7f2518131a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34632/0 (socket says 172.21.15.97:34632) 2024-03-27T08:42:01.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251d539640 1 -- 172.21.15.97:0/198556187 learned_addr learned my addr 172.21.15.97:0/198556187 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251cd38640 1 --2- 172.21.15.97:0/198556187 >> v2:172.21.15.97:3300/0 conn(0x7f25181378c0 0x7f2518139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f250ffff640 1 -- 172.21.15.97:0/198556187 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017550112 0 0) 0x7f25180a06a0 con 0x7f2518131650 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f250ffff640 1 -- 172.21.15.97:0/198556187 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2504003580 con 0x7f2518131650 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251cd38640 1 -- 172.21.15.97:0/198556187 >> v1:172.21.15.97:6789/0 conn(0x7f2518131650 legacy=0x7f2518131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.234+0000 7f251cd38640 1 -- 172.21.15.97:0/198556187 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251806de30 con 0x7f25181378c0 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f251cd38640 1 --2- 172.21.15.97:0/198556187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25181378c0 0x7f2518139ce0 crc :-1 s=READY pgs=13040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2431bfcbebb001e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f250ffff640 1 -- 172.21.15.97:0/198556187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2508009d90 con 0x7f25181378c0 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f250ffff640 1 -- 172.21.15.97:0/198556187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f250800e230 con 0x7f25181378c0 2024-03-27T08:42:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f250ffff640 1 -- 172.21.15.97:0/198556187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f250800e4d0 con 0x7f25181378c0 2024-03-27T08:42:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f251f79d640 1 -- 172.21.15.97:0/198556187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25181378c0 msgr2=0x7f2518139ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f251f79d640 1 --2- 172.21.15.97:0/198556187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25181378c0 0x7f2518139ce0 crc :-1 s=READY pgs=13040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f251f79d640 1 -- 172.21.15.97:0/198556187 shutdown_connections 2024-03-27T08:42:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f251f79d640 1 --2- 172.21.15.97:0/198556187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25181378c0 0x7f2518139ce0 unknown :-1 s=CLOSED pgs=13040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.235+0000 7f251f79d640 1 -- 172.21.15.97:0/198556187 >> 172.21.15.97:0/198556187 conn(0x7f25180a15e0 msgr2=0x7f25180a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.236+0000 7f251f79d640 1 -- 172.21.15.97:0/198556187 shutdown_connections 2024-03-27T08:42:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.236+0000 7f251f79d640 1 -- 172.21.15.97:0/198556187 wait complete. 2024-03-27T08:42:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.236+0000 7f251f79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.237+0000 7f251f79d640 1 Processor -- start 2024-03-27T08:42:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.237+0000 7f251f79d640 1 -- start start 2024-03-27T08:42:01.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.238+0000 7f251f79d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 0x7f251811e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:01.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.238+0000 7f251f79d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f251811e070 con 0x7f2518131650 2024-03-27T08:42:01.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.238+0000 7f251d539640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 0x7f251811e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.238+0000 7f251d539640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 0x7f251811e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41902/0 (socket says 172.21.15.97:41902) 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.238+0000 7f251d539640 1 -- 172.21.15.97:0/3927926013 learned_addr learned my addr 172.21.15.97:0/3927926013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f251d539640 1 -- 172.21.15.97:0/3927926013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2518120e50 con 0x7f2518131650 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f251d539640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 0x7f251811e4f0 crc :-1 s=READY pgs=13041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f25140070b0 con 0x7f2518131650 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2514007210 con 0x7f2518131650 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2518121bb0 con 0x7f2518131650 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f25140074b0 con 0x7f2518131650 2024-03-27T08:42:01.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.239+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25181218b0 con 0x7f2518131650 2024-03-27T08:42:01.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.240+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f2514007660 con 0x7f2518131650 2024-03-27T08:42:01.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.240+0000 7f250dffb640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24fc0387a0 0x7f24fc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:01.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.240+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2518073320 con 0x7f2518131650 2024-03-27T08:42:01.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.240+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f2514048cf0 con 0x7f2518131650 2024-03-27T08:42:01.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.240+0000 7f251cd38640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24fc0387a0 0x7f24fc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:01.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.241+0000 7f251cd38640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24fc0387a0 0x7f24fc03acb0 crc :-1 s=READY pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:01.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.245+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f25140092e0 con 0x7f2518131650 2024-03-27T08:42:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.420+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 --> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25181210a0 con 0x7f24fc0387a0 2024-03-27T08:42:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.423+0000 7f250dffb640 1 -- 172.21.15.97:0/3927926013 <== mgr.4099 v2:172.21.15.190:6800/3566337009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+107671 (crc 0 0 0) 0x7f25181210a0 con 0x7f24fc0387a0 2024-03-27T08:42:01.416 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T08:42:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24fc0387a0 msgr2=0x7f24fc03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24fc0387a0 0x7f24fc03acb0 crc :-1 s=READY pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 msgr2=0x7f251811e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 0x7f251811e4f0 crc :-1 s=READY pgs=13041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 shutdown_connections 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f24fc0387a0 0x7f24fc03acb0 unknown :-1 s=CLOSED pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 --2- 172.21.15.97:0/3927926013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2518131650 0x7f251811e4f0 unknown :-1 s=CLOSED pgs=13041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 >> 172.21.15.97:0/3927926013 conn(0x7f25180a15e0 msgr2=0x7f25181358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 shutdown_connections 2024-03-27T08:42:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.426+0000 7f251f79d640 1 -- 172.21.15.97:0/3927926013 wait complete. 2024-03-27T08:42:01.429 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":1275,"stamp":"2024-03-27T08:42:00.192670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":41,"num_shallow_scrub_errors":41,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":296092,"kb_used_data":14780,"kb_used_omap":0,"kb_used_meta":281280,"kb_avail":419134308,"statfs":{"total":429496729600,"available":429193531392,"internally_reserved":0,"allocated":15134720,"data_stored":6454419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":288030720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004150"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1278,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.861667+0000","last_change":"2024-03-27T08:41:39.886103+0000","last_active":"2024-03-27T08:41:39.861667+0000","last_peered":"2024-03-27T08:41:39.861667+0000","last_clean":"2024-03-27T08:41:39.861667+0000","last_became_active":"2024-03-27T08:27:58.944924+0000","last_became_peered":"2024-03-27T08:27:58.944924+0000","last_unstale":"2024-03-27T08:41:39.861667+0000","last_undegraded":"2024-03-27T08:41:39.861667+0000","last_fullsized":"2024-03-27T08:41:39.861667+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.885894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.885894+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.401141+0000","last_change":"2024-03-27T08:41:39.428243+0000","last_active":"2024-03-27T08:41:39.401141+0000","last_peered":"2024-03-27T08:41:39.401141+0000","last_clean":"2024-03-27T08:41:39.401141+0000","last_became_active":"2024-03-27T08:27:59.029389+0000","last_became_peered":"2024-03-27T08:27:59.029389+0000","last_unstale":"2024-03-27T08:41:39.401141+0000","last_undegraded":"2024-03-27T08:41:39.401141+0000","last_fullsized":"2024-03-27T08:41:39.401141+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.428008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.428008+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1264,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.321902+0000","last_change":"2024-03-27T08:41:40.347912+0000","last_active":"2024-03-27T08:41:40.321902+0000","last_peered":"2024-03-27T08:41:40.321902+0000","last_clean":"2024-03-27T08:41:40.321902+0000","last_became_active":"2024-03-27T08:27:58.934198+0000","last_became_peered":"2024-03-27T08:27:58.934198+0000","last_unstale":"2024-03-27T08:41:40.321902+0000","last_undegraded":"2024-03-27T08:41:40.321902+0000","last_fullsized":"2024-03-27T08:41:40.321902+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.347677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.347677+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.799273+0000","last_change":"2024-03-27T08:41:40.843528+0000","last_active":"2024-03-27T08:41:40.799273+0000","last_peered":"2024-03-27T08:41:40.799273+0000","last_clean":"2024-03-27T08:41:40.799273+0000","last_became_active":"2024-03-27T08:27:58.963619+0000","last_became_peered":"2024-03-27T08:27:58.963619+0000","last_unstale":"2024-03-27T08:41:40.799273+0000","last_undegraded":"2024-03-27T08:41:40.799273+0000","last_fullsized":"2024-03-27T08:41:40.799273+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.843316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.843316+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1278,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.858343+0000","last_change":"2024-03-27T08:41:39.895710+0000","last_active":"2024-03-27T08:41:39.858343+0000","last_peered":"2024-03-27T08:41:39.858343+0000","last_clean":"2024-03-27T08:41:39.858343+0000","last_became_active":"2024-03-27T08:27:58.967369+0000","last_became_peered":"2024-03-27T08:27:58.967369+0000","last_unstale":"2024-03-27T08:41:39.858343+0000","last_undegraded":"2024-03-27T08:41:39.858343+0000","last_fullsized":"2024-03-27T08:41:39.858343+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.895521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.895521+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.401776+0000","last_change":"2024-03-27T08:41:39.429857+0000","last_active":"2024-03-27T08:41:39.401776+0000","last_peered":"2024-03-27T08:41:39.401776+0000","last_clean":"2024-03-27T08:41:39.401776+0000","last_became_active":"2024-03-27T08:27:58.956201+0000","last_became_peered":"2024-03-27T08:27:58.956201+0000","last_unstale":"2024-03-27T08:41:39.401776+0000","last_undegraded":"2024-03-27T08:41:39.401776+0000","last_fullsized":"2024-03-27T08:41:39.401776+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.429665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.429665+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.397695+0000","last_change":"2024-03-27T08:41:39.443162+0000","last_active":"2024-03-27T08:41:39.397695+0000","last_peered":"2024-03-27T08:41:39.397695+0000","last_clean":"2024-03-27T08:41:39.397695+0000","last_became_active":"2024-03-27T08:27:58.981709+0000","last_became_peered":"2024-03-27T08:27:58.981709+0000","last_unstale":"2024-03-27T08:41:39.397695+0000","last_undegraded":"2024-03-27T08:41:39.397695+0000","last_fullsized":"2024-03-27T08:41:39.397695+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.442941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.442941+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.394065+0000","last_change":"2024-03-27T08:41:39.441693+0000","last_active":"2024-03-27T08:41:39.394065+0000","last_peered":"2024-03-27T08:41:39.394065+0000","last_clean":"2024-03-27T08:41:39.394065+0000","last_became_active":"2024-03-27T08:27:58.977694+0000","last_became_peered":"2024-03-27T08:27:58.977694+0000","last_unstale":"2024-03-27T08:41:39.394065+0000","last_undegraded":"2024-03-27T08:41:39.394065+0000","last_fullsized":"2024-03-27T08:41:39.394065+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.441499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.441499+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.797841+0000","last_change":"2024-03-27T08:41:40.845448+0000","last_active":"2024-03-27T08:41:40.797841+0000","last_peered":"2024-03-27T08:41:40.797841+0000","last_clean":"2024-03-27T08:41:40.797841+0000","last_became_active":"2024-03-27T08:27:58.979333+0000","last_became_peered":"2024-03-27T08:27:58.979333+0000","last_unstale":"2024-03-27T08:41:40.797841+0000","last_undegraded":"2024-03-27T08:41:40.797841+0000","last_fullsized":"2024-03-27T08:41:40.797841+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.845238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.845238+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1264,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.321082+0000","last_change":"2024-03-27T08:41:40.344168+0000","last_active":"2024-03-27T08:41:40.321082+0000","last_peered":"2024-03-27T08:41:40.321082+0000","last_clean":"2024-03-27T08:41:40.321082+0000","last_became_active":"2024-03-27T08:27:58.939180+0000","last_became_peered":"2024-03-27T08:27:58.939180+0000","last_unstale":"2024-03-27T08:41:40.321082+0000","last_undegraded":"2024-03-27T08:41:40.321082+0000","last_fullsized":"2024-03-27T08:41:40.321082+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.343899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.343899+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.793939+0000","last_change":"2024-03-27T08:41:40.837838+0000","last_active":"2024-03-27T08:41:40.793939+0000","last_peered":"2024-03-27T08:41:40.793939+0000","last_clean":"2024-03-27T08:41:40.793939+0000","last_became_active":"2024-03-27T08:27:58.955512+0000","last_became_peered":"2024-03-27T08:27:58.955512+0000","last_unstale":"2024-03-27T08:41:40.793939+0000","last_undegraded":"2024-03-27T08:41:40.793939+0000","last_fullsized":"2024-03-27T08:41:40.793939+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.837629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.837629+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.392038+0000","last_change":"2024-03-27T08:41:39.437300+0000","last_active":"2024-03-27T08:41:39.392038+0000","last_peered":"2024-03-27T08:41:39.392038+0000","last_clean":"2024-03-27T08:41:39.392038+0000","last_became_active":"2024-03-27T08:27:59.098593+0000","last_became_peered":"2024-03-27T08:27:59.098593+0000","last_unstale":"2024-03-27T08:41:39.392038+0000","last_undegraded":"2024-03-27T08:41:39.392038+0000","last_fullsized":"2024-03-27T08:41:39.392038+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.437100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.437100+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.792924+0000","last_change":"2024-03-27T08:41:40.841928+0000","last_active":"2024-03-27T08:41:40.792924+0000","last_peered":"2024-03-27T08:41:40.792924+0000","last_clean":"2024-03-27T08:41:40.792924+0000","last_became_active":"2024-03-27T08:27:58.973352+0000","last_became_peered":"2024-03-27T08:27:58.973352+0000","last_unstale":"2024-03-27T08:41:40.792924+0000","last_undegraded":"2024-03-27T08:41:40.792924+0000","last_fullsized":"2024-03-27T08:41:40.792924+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.841699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.841699+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.788255+0000","last_change":"2024-03-27T08:41:40.835255+0000","last_active":"2024-03-27T08:41:40.788255+0000","last_peered":"2024-03-27T08:41:40.788255+0000","last_clean":"2024-03-27T08:41:40.788255+0000","last_became_active":"2024-03-27T08:27:59.065991+0000","last_became_peered":"2024-03-27T08:27:59.065991+0000","last_unstale":"2024-03-27T08:41:40.788255+0000","last_undegraded":"2024-03-27T08:41:40.788255+0000","last_fullsized":"2024-03-27T08:41:40.788255+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.835038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.835038+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.388975+0000","last_change":"2024-03-27T08:41:39.440659+0000","last_active":"2024-03-27T08:41:39.388975+0000","last_peered":"2024-03-27T08:41:39.388975+0000","last_clean":"2024-03-27T08:41:39.388975+0000","last_became_active":"2024-03-27T08:27:59.007454+0000","last_became_peered":"2024-03-27T08:27:59.007454+0000","last_unstale":"2024-03-27T08:41:39.388975+0000","last_undegraded":"2024-03-27T08:41:39.388975+0000","last_fullsized":"2024-03-27T08:41:39.388975+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.440470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.440470+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.388287+0000","last_change":"2024-03-27T08:41:39.432341+0000","last_active":"2024-03-27T08:41:39.388287+0000","last_peered":"2024-03-27T08:41:39.388287+0000","last_clean":"2024-03-27T08:41:39.388287+0000","last_became_active":"2024-03-27T08:27:59.015211+0000","last_became_peered":"2024-03-27T08:27:59.015211+0000","last_unstale":"2024-03-27T08:41:39.388287+0000","last_undegraded":"2024-03-27T08:41:39.388287+0000","last_fullsized":"2024-03-27T08:41:39.388287+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.432122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.432122+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.385878+0000","last_change":"2024-03-27T08:41:39.435058+0000","last_active":"2024-03-27T08:41:39.385878+0000","last_peered":"2024-03-27T08:41:39.385878+0000","last_clean":"2024-03-27T08:41:39.385878+0000","last_became_active":"2024-03-27T08:27:58.960673+0000","last_became_peered":"2024-03-27T08:27:58.960673+0000","last_unstale":"2024-03-27T08:41:39.385878+0000","last_undegraded":"2024-03-27T08:41:39.385878+0000","last_fullsized":"2024-03-27T08:41:39.385878+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.434838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.434838+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.786743+0000","last_change":"2024-03-27T08:41:40.833194+0000","last_active":"2024-03-27T08:41:40.786743+0000","last_peered":"2024-03-27T08:41:40.786743+0000","last_clean":"2024-03-27T08:41:40.786743+0000","last_became_active":"2024-03-27T08:27:59.034744+0000","last_became_peered":"2024-03-27T08:27:59.034744+0000","last_unstale":"2024-03-27T08:41:40.786743+0000","last_undegraded":"2024-03-27T08:41:40.786743+0000","last_fullsized":"2024-03-27T08:41:40.786743+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.832937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.832937+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1264,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.319824+0000","last_change":"2024-03-27T08:41:40.349759+0000","last_active":"2024-03-27T08:41:40.319824+0000","last_peered":"2024-03-27T08:41:40.319824+0000","last_clean":"2024-03-27T08:41:40.319824+0000","last_became_active":"2024-03-27T08:27:58.988938+0000","last_became_peered":"2024-03-27T08:27:58.988938+0000","last_unstale":"2024-03-27T08:41:40.319824+0000","last_undegraded":"2024-03-27T08:41:40.319824+0000","last_fullsized":"2024-03-27T08:41:40.319824+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.349549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.349549+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1278,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.854259+0000","last_change":"2024-03-27T08:41:39.888543+0000","last_active":"2024-03-27T08:41:39.854259+0000","last_peered":"2024-03-27T08:41:39.854259+0000","last_clean":"2024-03-27T08:41:39.854259+0000","last_became_active":"2024-03-27T08:27:58.940428+0000","last_became_peered":"2024-03-27T08:27:58.940428+0000","last_unstale":"2024-03-27T08:41:39.854259+0000","last_undegraded":"2024-03-27T08:41:39.854259+0000","last_fullsized":"2024-03-27T08:41:39.854259+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.888337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.888337+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1278,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.856592+0000","last_change":"2024-03-27T08:41:39.892755+0000","last_active":"2024-03-27T08:41:39.856592+0000","last_peered":"2024-03-27T08:41:39.856592+0000","last_clean":"2024-03-27T08:41:39.856592+0000","last_became_active":"2024-03-27T08:27:58.982971+0000","last_became_peered":"2024-03-27T08:27:58.982971+0000","last_unstale":"2024-03-27T08:41:39.856592+0000","last_undegraded":"2024-03-27T08:41:39.856592+0000","last_fullsized":"2024-03-27T08:41:39.856592+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.892546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.892546+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1278,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.852447+0000","last_change":"2024-03-27T08:41:39.890919+0000","last_active":"2024-03-27T08:41:39.852447+0000","last_peered":"2024-03-27T08:41:39.852447+0000","last_clean":"2024-03-27T08:41:39.852447+0000","last_became_active":"2024-03-27T08:27:59.017861+0000","last_became_peered":"2024-03-27T08:27:59.017861+0000","last_unstale":"2024-03-27T08:41:39.852447+0000","last_undegraded":"2024-03-27T08:41:39.852447+0000","last_fullsized":"2024-03-27T08:41:39.852447+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.890703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.890703+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1264,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.318928+0000","last_change":"2024-03-27T08:41:40.348846+0000","last_active":"2024-03-27T08:41:40.318928+0000","last_peered":"2024-03-27T08:41:40.318928+0000","last_clean":"2024-03-27T08:41:40.318928+0000","last_became_active":"2024-03-27T08:27:58.942454+0000","last_became_peered":"2024-03-27T08:27:58.942454+0000","last_unstale":"2024-03-27T08:41:40.318928+0000","last_undegraded":"2024-03-27T08:41:40.318928+0000","last_fullsized":"2024-03-27T08:41:40.318928+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.348629+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.785539+0000","last_change":"2024-03-27T08:41:40.838874+0000","last_active":"2024-03-27T08:41:40.785539+0000","last_peered":"2024-03-27T08:41:40.785539+0000","last_clean":"2024-03-27T08:41:40.785539+0000","last_became_active":"2024-03-27T08:27:58.960185+0000","last_became_peered":"2024-03-27T08:27:58.960185+0000","last_unstale":"2024-03-27T08:41:40.785539+0000","last_undegraded":"2024-03-27T08:41:40.785539+0000","last_fullsized":"2024-03-27T08:41:40.785539+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.838661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.838661+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.781786+0000","last_change":"2024-03-27T08:41:40.829922+0000","last_active":"2024-03-27T08:41:40.781786+0000","last_peered":"2024-03-27T08:41:40.781786+0000","last_clean":"2024-03-27T08:41:40.781786+0000","last_became_active":"2024-03-27T08:27:58.969756+0000","last_became_peered":"2024-03-27T08:27:58.969756+0000","last_unstale":"2024-03-27T08:41:40.781786+0000","last_undegraded":"2024-03-27T08:41:40.781786+0000","last_fullsized":"2024-03-27T08:41:40.781786+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.829694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.829694+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.386972+0000","last_change":"2024-03-27T08:41:39.438619+0000","last_active":"2024-03-27T08:41:39.386972+0000","last_peered":"2024-03-27T08:41:39.386972+0000","last_clean":"2024-03-27T08:41:39.386972+0000","last_became_active":"2024-03-27T08:27:59.000404+0000","last_became_peered":"2024-03-27T08:27:59.000404+0000","last_unstale":"2024-03-27T08:41:39.386972+0000","last_undegraded":"2024-03-27T08:41:39.386972+0000","last_fullsized":"2024-03-27T08:41:39.386972+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.438422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.438422+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1258,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.779819+0000","last_change":"2024-03-27T08:41:40.821023+0000","last_active":"2024-03-27T08:41:40.779819+0000","last_peered":"2024-03-27T08:41:40.779819+0000","last_clean":"2024-03-27T08:41:40.779819+0000","last_became_active":"2024-03-27T08:27:59.004002+0000","last_became_peered":"2024-03-27T08:27:59.004002+0000","last_unstale":"2024-03-27T08:41:40.779819+0000","last_undegraded":"2024-03-27T08:41:40.779819+0000","last_fullsized":"2024-03-27T08:41:40.779819+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.820755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.820755+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.383090+0000","last_change":"2024-03-27T08:41:39.429107+0000","last_active":"2024-03-27T08:41:39.383090+0000","last_peered":"2024-03-27T08:41:39.383090+0000","last_clean":"2024-03-27T08:41:39.383090+0000","last_became_active":"2024-03-27T08:27:59.045303+0000","last_became_peered":"2024-03-27T08:27:59.045303+0000","last_unstale":"2024-03-27T08:41:39.383090+0000","last_undegraded":"2024-03-27T08:41:39.383090+0000","last_fullsized":"2024-03-27T08:41:39.383090+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.428884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.428884+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.382513+0000","last_change":"2024-03-27T08:41:39.433158+0000","last_active":"2024-03-27T08:41:39.382513+0000","last_peered":"2024-03-27T08:41:39.382513+0000","last_clean":"2024-03-27T08:41:39.382513+0000","last_became_active":"2024-03-27T08:27:58.964911+0000","last_became_peered":"2024-03-27T08:27:58.964911+0000","last_unstale":"2024-03-27T08:41:39.382513+0000","last_undegraded":"2024-03-27T08:41:39.382513+0000","last_fullsized":"2024-03-27T08:41:39.382513+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.432933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.432933+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1294,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.379930+0000","last_change":"2024-03-27T08:41:39.418460+0000","last_active":"2024-03-27T08:41:39.379930+0000","last_peered":"2024-03-27T08:41:39.379930+0000","last_clean":"2024-03-27T08:41:39.379930+0000","last_became_active":"2024-03-27T08:27:58.985998+0000","last_became_peered":"2024-03-27T08:27:58.985998+0000","last_unstale":"2024-03-27T08:41:39.379930+0000","last_undegraded":"2024-03-27T08:41:39.379930+0000","last_fullsized":"2024-03-27T08:41:39.379930+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.418245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.418245+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1291,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.850870+0000","last_change":"2024-03-27T08:41:39.894447+0000","last_active":"2024-03-27T08:41:39.850870+0000","last_peered":"2024-03-27T08:41:39.850870+0000","last_clean":"2024-03-27T08:41:39.850870+0000","last_became_active":"2024-03-27T08:27:58.934959+0000","last_became_peered":"2024-03-27T08:27:58.934959+0000","last_unstale":"2024-03-27T08:41:39.850870+0000","last_undegraded":"2024-03-27T08:41:39.850870+0000","last_fullsized":"2024-03-27T08:41:39.850870+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'7","last_scrub_stamp":"2024-03-27T08:41:39.894205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.894205+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1264,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.317850+0000","last_change":"2024-03-27T08:41:40.340929+0000","last_active":"2024-03-27T08:41:40.317850+0000","last_peered":"2024-03-27T08:41:40.317850+0000","last_clean":"2024-03-27T08:41:40.317850+0000","last_became_active":"2024-03-27T08:27:58.972299+0000","last_became_peered":"2024-03-27T08:27:58.972299+0000","last_unstale":"2024-03-27T08:41:40.317850+0000","last_undegraded":"2024-03-27T08:41:40.317850+0000","last_fullsized":"2024-03-27T08:41:40.317850+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.340670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.340670+0000","last_clean_scrub_stamp":"2024-03-27T08:27:57.746239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1867,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.849668+0000","last_change":"2024-03-27T08:41:39.884335+0000","last_active":"2024-03-27T08:41:39.849668+0000","last_peered":"2024-03-27T08:41:39.849668+0000","last_clean":"2024-03-27T08:41:39.849668+0000","last_became_active":"2024-03-27T08:23:09.061537+0000","last_became_peered":"2024-03-27T08:23:09.061537+0000","last_unstale":"2024-03-27T08:41:39.849668+0000","last_undegraded":"2024-03-27T08:41:39.849668+0000","last_fullsized":"2024-03-27T08:41:39.849668+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.884128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.884128+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1853,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.317682+0000","last_change":"2024-03-27T08:41:40.338686+0000","last_active":"2024-03-27T08:41:40.317682+0000","last_peered":"2024-03-27T08:41:40.317682+0000","last_clean":"2024-03-27T08:41:40.317682+0000","last_became_active":"2024-03-27T08:23:09.053982+0000","last_became_peered":"2024-03-27T08:23:09.053982+0000","last_unstale":"2024-03-27T08:41:40.317682+0000","last_undegraded":"2024-03-27T08:41:40.317682+0000","last_fullsized":"2024-03-27T08:41:40.317682+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.338428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.338428+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1859,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.779667+0000","last_change":"2024-03-27T08:41:40.822384+0000","last_active":"2024-03-27T08:41:40.779667+0000","last_peered":"2024-03-27T08:41:40.779667+0000","last_clean":"2024-03-27T08:41:40.779667+0000","last_became_active":"2024-03-27T08:23:09.046715+0000","last_became_peered":"2024-03-27T08:23:09.046715+0000","last_unstale":"2024-03-27T08:41:40.779667+0000","last_undegraded":"2024-03-27T08:41:40.779667+0000","last_fullsized":"2024-03-27T08:41:40.779667+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.822141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.822141+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1853,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.315563+0000","last_change":"2024-03-27T08:41:40.345149+0000","last_active":"2024-03-27T08:41:40.315563+0000","last_peered":"2024-03-27T08:41:40.315563+0000","last_clean":"2024-03-27T08:41:40.315563+0000","last_became_active":"2024-03-27T08:23:09.046892+0000","last_became_peered":"2024-03-27T08:23:09.046892+0000","last_unstale":"2024-03-27T08:41:40.315563+0000","last_undegraded":"2024-03-27T08:41:40.315563+0000","last_fullsized":"2024-03-27T08:41:40.315563+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:40.344903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.344903+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1895,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.379928+0000","last_change":"2024-03-27T08:41:39.391140+0000","last_active":"2024-03-27T08:41:39.379928+0000","last_peered":"2024-03-27T08:41:39.379928+0000","last_clean":"2024-03-27T08:41:39.379928+0000","last_became_active":"2024-03-27T08:23:09.056724+0000","last_became_peered":"2024-03-27T08:23:09.056724+0000","last_unstale":"2024-03-27T08:41:39.379928+0000","last_undegraded":"2024-03-27T08:41:39.379928+0000","last_fullsized":"2024-03-27T08:41:39.379928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.390817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.390817+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1867,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.848618+0000","last_change":"2024-03-27T08:41:39.879833+0000","last_active":"2024-03-27T08:41:39.848618+0000","last_peered":"2024-03-27T08:41:39.848618+0000","last_clean":"2024-03-27T08:41:39.848618+0000","last_became_active":"2024-03-27T08:23:09.049650+0000","last_became_peered":"2024-03-27T08:23:09.049650+0000","last_unstale":"2024-03-27T08:41:39.848618+0000","last_undegraded":"2024-03-27T08:41:39.848618+0000","last_fullsized":"2024-03-27T08:41:39.848618+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.879492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.879492+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1895,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.380020+0000","last_change":"2024-03-27T08:41:39.416537+0000","last_active":"2024-03-27T08:41:39.380020+0000","last_peered":"2024-03-27T08:41:39.380020+0000","last_clean":"2024-03-27T08:41:39.380020+0000","last_became_active":"2024-03-27T08:23:09.070092+0000","last_became_peered":"2024-03-27T08:23:09.070092+0000","last_unstale":"2024-03-27T08:41:39.380020+0000","last_undegraded":"2024-03-27T08:41:39.380020+0000","last_fullsized":"2024-03-27T08:41:39.380020+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.416320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.416320+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1867,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:39.849735+0000","last_change":"2024-03-27T08:41:39.873355+0000","last_active":"2024-03-27T08:41:39.849735+0000","last_peered":"2024-03-27T08:41:39.849735+0000","last_clean":"2024-03-27T08:41:39.849735+0000","last_became_active":"2024-03-27T08:23:09.060266+0000","last_became_peered":"2024-03-27T08:23:09.060266+0000","last_unstale":"2024-03-27T08:41:39.849735+0000","last_undegraded":"2024-03-27T08:41:39.849735+0000","last_fullsized":"2024-03-27T08:41:39.849735+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T08:41:39.873009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:39.873009+0000","last_clean_scrub_stamp":"2024-03-27T08:23:07.989354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":1992,"reported_epoch":714,"state":"active+clean+inconsistent","last_fresh":"2024-03-27T08:41:40.777643+0000","last_change":"2024-03-27T08:41:40.847062+0000","last_active":"2024-03-27T08:41:40.777643+0000","last_peered":"2024-03-27T08:41:40.777643+0000","last_clean":"2024-03-27T08:41:40.777643+0000","last_became_active":"2024-03-27T08:23:07.026654+0000","last_became_peered":"2024-03-27T08:23:07.026654+0000","last_unstale":"2024-03-27T08:41:40.777643+0000","last_undegraded":"2024-03-27T08:41:40.777643+0000","last_fullsized":"2024-03-27T08:41:40.777643+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'97","last_scrub_stamp":"2024-03-27T08:41:40.846789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T08:41:40.846789+0000","last_clean_scrub_stamp":"2024-03-27T08:23:05.980792+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":32,"num_shallow_scrub_errors":32,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":8,"num_shallow_scrub_errors":8,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":79508,"kb_used_data":3980,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":104778092,"statfs":{"total":107374182400,"available":107292766208,"internally_reserved":0,"allocated":4075520,"data_stored":1916648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":83128,"kb_used_data":3440,"kb_used_omap":0,"kb_used_meta":79680,"kb_avail":104774472,"statfs":{"total":107374182400,"available":107289059328,"internally_reserved":0,"allocated":3522560,"data_stored":1319557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81592320},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":66436,"kb_used_data":3388,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104791164,"statfs":{"total":107374182400,"available":107306151936,"internally_reserved":0,"allocated":3469312,"data_stored":1313805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":67020,"kb_used_data":3972,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":104790580,"statfs":{"total":107374182400,"available":107305553920,"internally_reserved":0,"allocated":4067328,"data_stored":1904409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-27T08:42:01.433 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-27T08:42:01.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.542+0000 7f70e019d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:01.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.550+0000 7f70e019d640 1 Processor -- start 2024-03-27T08:42:01.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.550+0000 7f70e019d640 1 -- start start 2024-03-27T08:42:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.551+0000 7f70e019d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70d812df60 0x7f70d8135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.551+0000 7f70e019d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d812d540 con 0x7f70d809c8b0 2024-03-27T08:42:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.551+0000 7f70e019d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f70d806e370 con 0x7f70d812df60 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.551+0000 7f70ddf39640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f70d809c8b0 0x7f70d812d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34648/0 (socket says 172.21.15.97:34648) 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.551+0000 7f70ddf39640 1 -- 172.21.15.97:0/2138176678 learned_addr learned my addr 172.21.15.97:0/2138176678 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.551+0000 7f70dd738640 1 --2- 172.21.15.97:0/2138176678 >> v2:172.21.15.97:3300/0 conn(0x7f70d812df60 0x7f70d8135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dd738640 1 -- 172.21.15.97:0/2138176678 >> v1:172.21.15.97:6789/0 conn(0x7f70d809c8b0 legacy=0x7f70d812d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dcf37640 1 -- 172.21.15.97:0/2138176678 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 355614451 0 0) 0x7f70d812d540 con 0x7f70d809c8b0 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dd738640 1 -- 172.21.15.97:0/2138176678 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d806ec50 con 0x7f70d812df60 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dd738640 1 --2- 172.21.15.97:0/2138176678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d812df60 0x7f70d8135cc0 crc :-1 s=READY pgs=13043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63d7e2f855b7ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dcf37640 1 -- 172.21.15.97:0/2138176678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f70c8009d90 con 0x7f70d812df60 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dcf37640 1 -- 172.21.15.97:0/2138176678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f70c800e230 con 0x7f70d812df60 2024-03-27T08:42:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.552+0000 7f70dcf37640 1 -- 172.21.15.97:0/2138176678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f70c800e4d0 con 0x7f70d812df60 2024-03-27T08:42:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 -- 172.21.15.97:0/2138176678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d812df60 msgr2=0x7f70d8135cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 --2- 172.21.15.97:0/2138176678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d812df60 0x7f70d8135cc0 crc :-1 s=READY pgs=13043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 -- 172.21.15.97:0/2138176678 shutdown_connections 2024-03-27T08:42:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 --2- 172.21.15.97:0/2138176678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d812df60 0x7f70d8135cc0 unknown :-1 s=CLOSED pgs=13043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 -- 172.21.15.97:0/2138176678 >> 172.21.15.97:0/2138176678 conn(0x7f70d80a06d0 msgr2=0x7f70d80a0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 -- 172.21.15.97:0/2138176678 shutdown_connections 2024-03-27T08:42:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 1 -- 172.21.15.97:0/2138176678 wait complete. 2024-03-27T08:42:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.553+0000 7f70e019d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-27T08:42:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.554+0000 7f70e019d640 1 Processor -- start 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.554+0000 7f70e019d640 1 -- start start 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.555+0000 7f70e019d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 0x7f70d811ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.555+0000 7f70e019d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70d811dfc0 con 0x7f70d809c8b0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.555+0000 7f70ddf39640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 0x7f70d811ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.555+0000 7f70ddf39640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 0x7f70d811ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41918/0 (socket says 172.21.15.97:41918) 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.555+0000 7f70ddf39640 1 -- 172.21.15.97:0/2531636634 learned_addr learned my addr 172.21.15.97:0/2531636634 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70ddf39640 1 -- 172.21.15.97:0/2531636634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d812acf0 con 0x7f70d809c8b0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70ddf39640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 0x7f70d811ba70 crc :-1 s=READY pgs=13044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f70d4004b80 con 0x7f70d809c8b0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f70d4004ce0 con 0x7f70d809c8b0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d811e250 con 0x7f70d809c8b0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f70d4007120 con 0x7f70d809c8b0 2024-03-27T08:42:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.556+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d812a990 con 0x7f70d809c8b0 2024-03-27T08:42:01.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.557+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d8063450 con 0x7f70d809c8b0 2024-03-27T08:42:01.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.557+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 45742+0+0 (crc 0 0 0) 0x7f70d40072d0 con 0x7f70d809c8b0 2024-03-27T08:42:01.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.557+0000 7f70ce7fc640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f70bc0387a0 0x7f70bc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T08:42:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.557+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(714..714 src has 652..714) ==== 3403+0+0 (crc 0 0 0) 0x7f70d4048440 con 0x7f70d809c8b0 2024-03-27T08:42:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.557+0000 7f70dd738640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f70bc0387a0 0x7f70bc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T08:42:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.558+0000 7f70dd738640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f70bc0387a0 0x7f70bc03acb0 crc :-1 s=READY pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T08:42:01.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.562+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186293 (crc 0 0 0) 0x7f70d400a9e0 con 0x7f70d809c8b0 2024-03-27T08:42:01.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.738+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f70d8069070 con 0x7f70d809c8b0 2024-03-27T08:42:01.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.741+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (crc 0 0 0) 0x7f70d400a5d0 con 0x7f70d809c8b0 2024-03-27T08:42:01.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.742+0000 7f70ce7fc640 1 -- 172.21.15.97:0/2531636634 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(1 keys) ==== 35+0+0 (crc 0 0 0) 0x7f70d400a9e0 con 0x7f70d809c8b0 2024-03-27T08:42:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.743+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f70bc0387a0 msgr2=0x7f70bc03acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.743+0000 7f70e019d640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f70bc0387a0 0x7f70bc03acb0 crc :-1 s=READY pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.743+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 msgr2=0x7f70d811ba70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 0x7f70d811ba70 crc :-1 s=READY pgs=13044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 shutdown_connections 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.190:6800/3566337009,v1:172.21.15.190:6801/3566337009] conn(0x7f70bc0387a0 0x7f70bc03acb0 unknown :-1 s=CLOSED pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 --2- 172.21.15.97:0/2531636634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70d809c8b0 0x7f70d811ba70 unknown :-1 s=CLOSED pgs=13044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 >> 172.21.15.97:0/2531636634 conn(0x7f70d80a06d0 msgr2=0x7f70d809d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 shutdown_connections 2024-03-27T08:42:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T08:42:01.744+0000 7f70e019d640 1 -- 172.21.15.97:0/2531636634 wait complete. 2024-03-27T08:42:01.749 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-27T08:42:01.749 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-27T08:42:01.749 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-27T08:42:01.750 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:42:01.837 INFO:tasks.ceph.osd.0:Stopped 2024-03-27T08:42:01.838 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-27T08:42:01.838 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:42:01.921 INFO:tasks.ceph.osd.1:Stopped 2024-03-27T08:42:01.922 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-27T08:42:01.922 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:42:01.998 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:42:02.005+0000 7f3d97672640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T08:42:01.998 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:42:02.005+0000 7f3d97672640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T08:42:01.998 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-03-27T08:42:02.005+0000 7f3d97672640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T08:42:02.002 INFO:tasks.ceph.osd.2:Stopped 2024-03-27T08:42:02.002 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-27T08:42:02.002 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:42:02.074 INFO:tasks.ceph.osd.3:Stopped 2024-03-27T08:42:02.074 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-27T08:42:02.075 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-27T08:42:02.075 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:42:02.102 INFO:tasks.ceph.mgr.x:Stopped 2024-03-27T08:42:02.102 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-27T08:42:02.102 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-27T08:42:02.102 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T08:42:02.116 INFO:tasks.ceph.mon.a:Stopped 2024-03-27T08:42:02.116 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-27T08:42:02.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-27T08:42:02.189 INFO:teuthology.orchestra.run.smithi097.stdout:2024-03-27T08:41:44.889929+0000 mon.a (mon.0) 1353 : cluster [ERR] Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-27T08:42:02.190 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-27T08:42:02.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-27T08:42:02.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-27T08:42:02.329 INFO:teuthology.orchestra.run.smithi097.stdout:2024-03-27T08:41:44.889929+0000 mon.a (mon.0) 1353 : cluster [ERR] Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-27T08:42:02.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:42:02.329 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-27T08:42:02.519 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:42:02.520 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-27T08:42:02.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:42:02.611 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-27T08:42:02.709 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:42:02.709 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-27T08:42:02.805 INFO:tasks.ceph:Archiving mon data... 2024-03-27T08:42:02.806 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/data/mon.a.tgz 2024-03-27T08:42:02.806 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-27T08:42:02.822 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.QpIgnmJlMX 2024-03-27T08:42:02.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QpIgnmJlMX 2024-03-27T08:42:02.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.QpIgnmJlMX 2024-03-27T08:42:03.037 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.QpIgnmJlMX is 273KB 2024-03-27T08:42:03.132 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.QpIgnmJlMX 2024-03-27T08:42:03.147 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-27T08:42:03.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-27T08:42:03.189 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-27T08:42:03.269 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-27T08:42:03.285 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-27T08:42:03.286 INFO:tasks.ceph:Archiving crash dumps... 2024-03-27T08:42:03.287 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/remote/smithi097/crash 2024-03-27T08:42:03.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-27T08:42:03.336 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/remote/smithi190/crash 2024-03-27T08:42:03.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-27T08:42:03.389 INFO:tasks.ceph:Compressing logs... 2024-03-27T08:42:03.389 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T08:42:03.392 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30823.log 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30903.log 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30943.log 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30823.log: /var/log/ceph/ceph-client.admin.30903.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30823.log.gz 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30943.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30903.log.gz 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-27T08:42:03.427 INFO:teuthology.orchestra.run.smithi190.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.30943.log.gz 2024-03-27T08:42:03.428 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30863.log 2024-03-27T08:42:03.428 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-27T08:42:03.428 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30863.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.30863.log.gz 2024-03-27T08:42:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100579.log 2024-03-27T08:42:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99184.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100579.log: /var/log/ceph/ceph-client.admin.99184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99268.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100579.log.gz 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99184.log.gz 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32511.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97531.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32511.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99268.log.gz 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31197.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32511.log.gz 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97480.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32514.log 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97480.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.31197.log.gz 2024-03-27T08:42:03.485 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97531.log.gz 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32642.log 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32513.log 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.97480.log.gz 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.32514.log.gz 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99678.log 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97927.log 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32642.log.gz 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32513.log: /var/log/ceph/ceph-client.admin.99678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99443.log 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32513.log.gz 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97927.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99678.log.gz 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32344.log 2024-03-27T08:42:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98251.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32344.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99443.log.gz 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99159.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.32344.log.gz 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.98251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99938.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.97927.log.gz 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100537.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99159.log: /var/log/ceph/ceph-client.admin.99938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32160.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.99159.log.gz 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100537.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.99938.log.gz 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99226.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.100537.log.gz 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100369.log 2024-03-27T08:42:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32160.log: /var/log/ceph/ceph-client.admin.99226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97583.log 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32160.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99331.log 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99226.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.100369.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100117.log 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34510.log 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99331.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.98251.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100117.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97583.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99630.log 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99331.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100117.log.gz 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96758.log 2024-03-27T08:42:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99783.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.99630.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32343.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99247.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99783.log: /var/log/ceph/ceph-client.admin.32343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97666.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99783.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32343.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99580.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97667.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97666.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99247.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97665.log 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96758.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97667.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.99580.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr: 90.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.97666.log.gz 2024-03-27T08:42:03.489 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32916.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99373.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97667.log.gz 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100453.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96825.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.32916.log.gz 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99373.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97665.log.gz 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99205.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99373.log.gz 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32748.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96825.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100453.log.gz 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99489.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99205.log.gz 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32512.log 2024-03-27T08:42:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99489.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32748.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.99489.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97582.log 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.96825.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100201.log 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.32512.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100495.log 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100201.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97582.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.100201.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98752.log 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97211.log 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.100495.log.gz 2024-03-27T08:42:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.98752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97418.log 2024-03-27T08:42:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99399.log 2024-03-27T08:42:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97290.log 2024-03-27T08:42:03.492 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.34510.log.gz 2024-03-27T08:42:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99399.log: gzip -5 --verbose 91.3% -- replaced with /var/log/ceph/ceph-client.admin.97211.log.gz 2024-03-27T08:42:03.492 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.97437.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.99399.log.gz 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96343.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97581.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr: 51.6% -- replaced with /var/log/ceph/ceph-client.admin.97437.log.gz 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.97418.log.gz 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97842.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32345.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97581.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.98752.log.gz 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34452.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:gzip 90.4% -5/var/log/ceph/ceph-client.admin.32345.log: --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97581.log.gz 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99464.log 2024-03-27T08:42:03.493 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32345.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97842.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99097.log 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.99464.log: gzip -5 92.0% -- replaced with /var/log/ceph/ceph-client.admin.96343.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32346.log 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34452.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99464.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97584.log 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.99097.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97441.log 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97584.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32346.log.gz 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96305.log 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97844.log 2024-03-27T08:42:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97441.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.97584.log.gz 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.97441.log.gz 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99534.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96305.log.gz 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97806.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr: 90.2%/var/log/ceph/ceph-client.admin.99534.log: -- replaced with /var/log/ceph/ceph-client.admin.97844.log.gz 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98540.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.99534.log.gz 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100327.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5/var/log/ceph/ceph-client.admin.98540.log: --verbose -- /var/log/ceph/ceph-client.admin.99913.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100327.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97806.log.gz 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32012.log 2024-03-27T08:42:03.495 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.100327.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.99913.log: -- /var/log/ceph/ceph-client.admin.97843.log 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32012.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.99913.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96566.log 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.32012.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.97843.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97530.log 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96566.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.97843.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99825.log 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32641.log 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97530.log: /var/log/ceph/ceph-client.admin.99825.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.97530.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99825.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34418.log 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.98540.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32641.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.96566.log.gz 2024-03-27T08:42:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34567.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.32641.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.97290.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97769.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99962.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97769.log: 90.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97975.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34418.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34567.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97733.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5/var/log/ceph/ceph-client.admin.97975.log: --verbose 86.3% -- replaced with /var/log/ceph/ceph-client.admin.99962.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97769.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.99762.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97733.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.97975.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99558.log 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99762.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97733.log.gz 2024-03-27T08:42:03.497 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.99657.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99558.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99762.log.gz 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99352.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99558.log.gz 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99741.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99657.log.gz 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97734.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.99741.log: -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99352.log.gz 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96527.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97734.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99741.log.gz 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99720.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.97734.log.gz 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97442.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32662.log 2024-03-27T08:42:03.498 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99720.log.gz 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32202.log 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32068.log 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96527.log.gz/var/log/ceph/ceph-client.admin.32662.log: 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99804.log 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32662.log.gz 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97442.log.gz 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- 86.3% /var/log/ceph/ceph-client.admin.33646.log 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32202.log.gz 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32068.log: /var/log/ceph/ceph-client.admin.99804.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32068.log.gz 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99699.log 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:gzip 87.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.99804.log.gz -- 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.97529.log 2024-03-27T08:42:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33646.log: /var/log/ceph/ceph-client.admin.99699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99398.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97529.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99699.log.gz 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100661.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99398.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97992.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97529.log.gz 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99398.log.gz 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33177.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.100661.log.gz 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100159.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97992.log: /var/log/ceph/ceph-client.admin.33177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97478.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32244.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97478.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.100159.log.gz 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97478.log.gz 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32787.log 2024-03-27T08:42:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32958.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32787.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32244.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32787.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96682.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32958.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32958.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.97992.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97770.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99310.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97807.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96682.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.97770.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100411.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32874.log 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99310.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100411.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.97807.log.gz 2024-03-27T08:42:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97479.log 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% 86.5% -- replaced with /var/log/ceph/ceph-client.admin.100411.log.gz 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33177.log.gz 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32874.log: /var/log/ceph/ceph-client.admin.97479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100243.log 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100621.log 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.32874.log.gz 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.97479.log.gz/var/log/ceph/ceph-client.admin.100243.log: 2024-03-27T08:42:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99490.log 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.100243.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96876.log 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100621.log: /var/log/ceph/ceph-client.admin.99490.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.100621.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99490.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32788.log 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96876.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30933.log 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32788.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32788.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.33646.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32118.log 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.96876.log.gz 2024-03-27T08:42:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34471.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.30933.log.gz 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100285.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32118.log.gz 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98204.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.100285.log: gzip -5 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34471.log.gz 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98951.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.100285.log.gz 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.98204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99846.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.98951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34548.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96701.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99846.log.gz 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97443.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96701.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34548.log.gz 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31169.log 2024-03-27T08:42:03.504 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96701.log.gz 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97443.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.97443.log.gz 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97895.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98433.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97895.log: /var/log/ceph/ceph-client.admin.31169.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.31169.log.gz 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97895.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99868.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.98433.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.98204.log.gz 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99606.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.98433.log.gz 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96233.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99289.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99868.log.gz 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr: 86.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99123.log 2024-03-27T08:42:03.505 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99606.log.gz 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96233.log: /var/log/ceph/ceph-client.admin.99289.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99289.log.gz 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96739.log 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.99123.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28613.log 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96739.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.99123.log.gz 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31012.log 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.98951.log.gz/var/log/ceph/ceph.tmp-client.admin.28613.log: 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28613.log.gz 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr: 91.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97580.log 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96739.log.gz 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99889.log 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.31012.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96720.log 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:42:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.97580.log: /var/log/ceph/ceph-client.admin.99889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100075.log 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.96720.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.99889.log.gz 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97580.log.gz 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98919.log 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5%/var/log/ceph/ceph-client.admin.100075.log: -- replaced with /var/log/ceph/ceph-client.admin.96720.log.gz 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.100075.log.gz 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.98919.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98919.log.gz 2024-03-27T08:42:03.507 INFO:teuthology.orchestra.run.smithi097.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.96233.log.gz 2024-03-27T08:42:03.632 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.2.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-27T08:42:03.860 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-03-27T08:42:04.900 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-27T08:42:08.186 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-27T08:42:08.191 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-27T08:42:08.395 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-27T08:42:08.396 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-03-27T08:42:08.396 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m4.991s 2024-03-27T08:42:08.396 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m9.542s 2024-03-27T08:42:08.397 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.425s 2024-03-27T08:42:08.494 INFO:teuthology.orchestra.run.smithi097.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-27T08:42:08.496 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T08:42:08.496 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m5.093s 2024-03-27T08:42:08.496 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m11.370s 2024-03-27T08:42:08.496 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.575s 2024-03-27T08:42:08.496 INFO:tasks.ceph:Archiving logs... 2024-03-27T08:42:08.497 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/remote/smithi097/log 2024-03-27T08:42:08.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T08:42:10.887 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/remote/smithi190/log 2024-03-27T08:42:10.888 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T08:42:12.376 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-27T08:42:12.385 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-27T08:42:12.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T08:42:12.388 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T08:42:12.487 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-27T08:42:12.496 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-27T08:42:12.496 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:42:12.499 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T08:42:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:42:12.515 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-03-27T08:42:12.518 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:42:12.519 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-27T08:42:12.519 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 +163us[ +160us] +/- 94ms 2024-03-27T08:42:12.519 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -469us[ -472us] +/- 76ms 2024-03-27T08:42:12.519 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +348us[ +345us] +/- 78ms 2024-03-27T08:42:12.519 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:42:12.520 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T08:42:12.520 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-03-27T08:42:12.520 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 +183us[ +162us] +/- 94ms 2024-03-27T08:42:12.520 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -461us[ -481us] +/- 76ms 2024-03-27T08:42:12.520 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +361us[ +340us] +/- 78ms 2024-03-27T08:42:12.520 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T08:42:12.520 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-27T08:42:12.529 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-27T08:42:12.530 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-27T08:42:12.539 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T08:42:12.630 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T08:42:12.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:42:12.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:42:12.729 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T08:42:12.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:42:12.760 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-27T08:42:12.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-27T08:42:12.779 INFO:teuthology.task.internal:Duration was 1627.638532 seconds 2024-03-27T08:42:12.779 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-27T08:42:12.788 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-27T08:42:12.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T08:42:12.791 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T08:42:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:42:12.844 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T08:42:13.134 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-27T08:42:13.135 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T08:42:13.135 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T08:42:13.156 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-03-27T08:42:13.157 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T08:42:13.202 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-27T08:42:13.202 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T08:42:13.204 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T08:42:13.272 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-27T08:42:13.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T08:42:13.274 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T08:42:13.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-27T08:42:13.420 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-27T08:42:13.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T08:42:13.448 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T08:42:13.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-27T08:42:13.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:42:13.491 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:42:13.513 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-03-27T08:42:13.541 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-03-27T08:42:13.561 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:42:13.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:42:13.583 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T08:42:13.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T08:42:13.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-27T08:42:13.626 INFO:teuthology.task.internal:Transferring archived files... 2024-03-27T08:42:13.627 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/remote/smithi097 2024-03-27T08:42:13.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T08:42:13.679 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880/remote/smithi190 2024-03-27T08:42:13.680 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T08:42:13.982 INFO:teuthology.task.internal:Removing archive directory... 2024-03-27T08:42:13.982 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T08:42:13.986 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T08:42:14.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-27T08:42:14.018 INFO:teuthology.task.internal:Not uploading archives. 2024-03-27T08:42:14.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-27T08:42:14.028 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-27T08:42:14.028 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T08:42:14.045 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T08:42:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 08:42 /home/ubuntu/cephtest 2024-03-27T08:42:14.059 INFO:teuthology.orchestra.run.smithi190.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 08:42 /home/ubuntu/cephtest 2024-03-27T08:42:14.061 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-27T08:42:14.074 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-27T08:42:14.095 INFO:teuthology.nuke:Checking targets against current locks 2024-03-27T08:42:14.122 DEBUG:teuthology.nuke:shortname: smithi097 2024-03-27T08:42:14.123 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T08:42:14.145 DEBUG:teuthology.nuke:shortname: smithi190 2024-03-27T08:42:14.145 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T08:42:14.149 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:07:02.055197', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxONJN7jzLFKzFOxCdVzc2pZrWcixdRHg3DKGiCS7tp2vvp/OUOaK15GFzgk/QKd5LXJ61keemHHi3aHNLkbGo='} 2024-03-27T08:42:14.176 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-03-27_08:05:55-crimson-rados-wip-matanb-crimson-foreign-ptr-distro-crimson-smithi/7625880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 08:07:02.056697', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMqxg1zoRJJjLS/Y+WaGnlKi1sat+8mnVePItmpNfEkGm/mIKFGUIDBacx6oEa13Ls9V4E+gXhM81ybdFzRl5QA='} 2024-03-27T08:42:14.201 INFO:teuthology.orchestra.console.smithi097:Power off 2024-03-27T08:42:14.201 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T08:42:14.226 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-03-27T08:42:14.226 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:14.243 INFO:teuthology.orchestra.console.smithi190:Power off 2024-03-27T08:42:14.243 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T08:42:14.253 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-27T08:42:14.266 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2024-03-27T08:42:14.266 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:14.293 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-03-27T08:42:18.254 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:18.294 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:18.365 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-27T08:42:18.405 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-03-27T08:42:22.367 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:22.405 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:22.478 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-03-27T08:42:22.517 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-03-27T08:42:22.579 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-03-27T08:42:26.522 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T08:42:26.633 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2024-03-27T08:42:26.734 INFO:teuthology.orchestra.console.smithi190:Power off completed 2024-03-27T08:42:26.838 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 1627.6385324001312 failure_reason: '"2024-03-27T08:41:44.889929+0000 mon.a (mon.0) 1353 : cluster [ERR] Health check failed: 41 scrub errors (OSD_SCRUB_ERRORS)" in cluster log' flavor: crimson owner: scheduled_matan@teuthology success: false 2024-03-27T08:42:26.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T08:42:26.926 INFO:teuthology.run:FAIL